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| A FEW POINTS ABOUT AXIOMATICS 
OF EDUCATION 


Vincentas Lamanauskas 
Vilnius University, Lithuania 
E-mail: vincentas. lamanauskas@sa.vu. It 


Postulates and axioms have been used as a means of knowledge since ancient times. 
The well-known ancient Greek thinkers Aristotle, Plato, Euclid, and others used this cognitive 
tool. As an example, we all know — Euclid’s postulates. Although mathematicians consider 
him the pioneer of mathematics, one of the most famous mathematicians, nevertheless, it is 
more appropriate to consider Euclid as a Greek philosopher, who presented the most important 
postulates and axioms in his book "Elements". The prevailing opinion is that in essence it 
is exactly enough to prove all mathematical statements. Even though that the most perfect 
axiomatics is classical Euclidian geometry, there is an alternative here, i.e., modern non- 
Euclidean geometry. 

Can such an analogy be found in education (pedagogy). Do we have educational 
alternatives? It turns out it is possible. Most of the educational statements we know can 
indeed be considered postulates and axioms. It is obvious that this is especially evident in folk 
pedagogy (ethnopedagogy). It seems to be saturated with postulates and axioms (proverbs, 
sayings, expressions, etc.). So, it can be said that: 


e Most sciences, such as logic, natural sciences widely use postulates and axioms 
as a cognitive instrument; 

e Postulates and axioms of education is a poorly theoretically analysed area; 

e Axiomatics of education is a rather spontaneous area of education. 


In order to analyse and get to know the axiomatics of education in more detail, first of 
all, it is necessary to understand the general foundations of postulates and axioms. 
Four essential points can be mentioned: 

e Postulates and axioms do not fundamentally differ at the level of modern 
understanding. Philosophers and logicians consider these two concepts 
synonymous. Therefore, in principle, we do not aim here to examine the 
differences between these concepts. Postulates and axioms have a historically 
formed context. Modern science understands postulates as initial statements of 
any theory, which do not need to be proven; 


e Postulates (axioms) are treated as obvious, unprovable truths because they are the 
result of long-term and repeated observations of phenomena; 

e Axiomatic method is characterised by precision, strict rules, often formalised 
language, although it is understood that not always and not everything can be 
formalised; 

e In education, the axiomatic method requires empirical confirmation. In this view, 


axioms are examined as hypotheses. 


The essential task of educational axiomatics can be represented in the following sequence 
(Figure 1). 
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Figure 1 625 
Relationship: Educational Axiomatics-Educational Theory-Educational Practice 


Educational 
practice (methodics 
and technologies) 


Postulates of 
education (axioms) 


Educational theory 
(-ies) 


Ethnopedagogy has accumulated many different postulates, which, by the way, have 
been verified by the long-term practice of many generations. Therefore, it is an urgent task not 
only to know them, but to systematise them. When the postulates are presented in a certain 
system, with a certain consistency, then it helps us to have an excellent methodological basis for 
any theory of education. From this point of view, the axiomatics of education and systemology 
of education are closely related fields determining one another. A cognitive field is developing 
systematic axiomatics (Lamanauskas, 2004). A question may arise as to how postulates (axioms) 
interact with laws and regularities of education, various rules, and principles of education. It 
would be wrong to think that they change the laws and regularities. Modern science tends 
to treat postulates (axioms) as a component supplementing laws and regularities. However, it 
cannot be denied that it is an important and fundamental methodological basis of education. 
The most important thing is that postulate (axiom) should perform an essential function — it 
would form pre-requisites for further conclusions (derived statements, principles, rules, etc.). It 
is obvious that axioms are the cornerstones of any theory. Of course, a scientific theory can be 
constructed without axioms, however, in scientific researchers’ opinion, such theory will not be 
of full value. It can be conditionally stated that until now the concepts of educational science 
have been constructed without "laying" axioms in their foundations. By using the possibilities 
provided by the computer, it was clearly established the place of axioms in the pedagogical 
theory structure. It turned out that axioms are not only the basis of scientific theory, but also 
the result, the level reached (Podlasyj, 1991). It can be said that axioms at the same time crown 
a scientific theory. Science education is special from this point of view because it has most 
abundant empirical base (whole "mountains" of facts have been accumulated). How not to get 
lost in such an abundance of educational information? How to sort out this abundance at least 
a little (separate the chaff from the grain)? This is where we cannot do without axiomatics. It 
remains only to wish researchers and educational practitioners to look into these issues much 
more deeply and in detail than they have so far. It is not so easy to find axioms in the abundance 
of empirical statements. Not every statement as a matter of fact can be an axiom. For this, it 
is necessary that the meaning of that statement remains unchanged in any educational context 
(both in terms of time and space). Only then, it is possible to construct various systems of 
educational axioms. 

When examining the issues of axiomatics of education, the concept of "theorem" 
is inevitably mentioned (Figure 2). Theorem (gr. theorema — teaching, rule) — a theoretical 
statement, the truth of which can be or is logically proven, based on previously established 
scientific statements (TZZ, 1985). 
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Figure 2 
Theorem of Education as a Postulate Outcome and as a Prerequisite of Rule Formulation 


Postulates of Theorems of Rules, principles, etc. of 


education (axioms) education education 


Two examples can be given that illustrate this connection. 
The first example 


Play — a natural child’s need (axiom) — if we create conditions for the child to play, then 
his development will be effective (theorem) — various game methodologies, game practice 
(didactic/hodegetic rules, and principles). 


The second example 


A child becomes independent by acting practically (axiom) — in order to develop the 
child’s independence, it is necessary to assign specific practical tasks and obligations to children 
(theorem) — various methodologies of independence education, practice of such education 
(didactic/hodegetic rules, and principles). 

It is necessary to note that each educational paradigm is based on certain essential 
postulates. The latter are which basically show the content of a specific paradigm and its 
coordinates in space and time. As an example, we can examine the postulates of the traditionalist- 
conservative educational paradigm. As Laty§ (2000) rightly noticed, this paradigm does not go 
to the past in any way, on the contrary, it often even regains its position in current and maybe 
in future education. It can be said that this paradigm is resistant to various new educational 
currents, educational fashions, and political change alternatives. 


Postulate 1: education provides students with basic knowledge, abilities and skills (axiom) 
— education is effective when priority is given to mastering the main elements of the 
teaching instrument (theorem) — it is important to teach children to read, write, and count, 
to provide other most necessary mathematical, natural science skills, and socio-humanitarian 
knowledge, abilities, skills, which are the basis for mastering any other academic subjects 
(didactic goals and tasks). 


Postulate 2: in the productive education process, only important and necessary knowledge 
is conveyed (axiom) — education process is effective when it focuses on the most significant 
areas of human cognition (theorem) — in the education process, it is necessary to distinguish 
fundamental things from superficial, indisputable things from temporary, passing, etc. 
(didactic goals and tasks). 


Postulate 3: eternal human values — the basis of education (axiom) — a morally saturated 
educational process enables the development of a strong, mature personality (theorem) — in 
the educational process, the value component is important, value orientation of the learners, 
the development of value attitudes and beliefs, all that preserves value and significance not 
only in the present but also in the future (didactic goals and tasks). 
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Here we have shown the postulates of only one of the paradigms. It is obvious that many 
proofs can be presented for each of them (classical didactics, philosophy, etc.). 

The 21* century insisted on qualitatively different education. Current global problems 
require a new educational paradigm. In other words, flexible educational systems require flexible 
pedagogy (Neimatoy, 2002). Rapid changes in education at all levels of the system require not 
only adequate operation, but first of all, adequate understanding of the ongoing processes. The 
main axiom — any quick and easy educational achievements are an illusion. Perhaps the most 
important education law in this aspect is - do not harm. 

It is obvious that 21* century requires qualitatively new competencies. The most essential 
can be represented in the form of a triad (Figure 3). 


Figure 3 
The Triad of Most Important Competencies 


Ability to act Ability to be 


Ability to live 


Although 20" century shaped technical progress (a new world view), on the other hand, 
it caused many undesirable phenomena. It is obvious that the reason lies not in the technique 
itself or in the technique progress, but in the insufficient understanding of its essence. The 
human factor of technical progress becomes the central axis. Researchers emphasize that a 
modern specialist must be both a professional and a humanist (Broks, 2000; Lamanauskas, 
2003). However, the man of the 21 century found himself on the boundary between reason 
and faith, truth and lies, illusions and myths. Therefore, it is understandable why the last decade 
of the 20" century could be characterised as a period of active search for new educational 
paradigms. The issue of the reconstruction of education in a humanistic and humanitarian 
direction (humanitarian and humanistic paradigms of education) is being discussed very 
actively. This approach to education today may seem to reject teaching and upbringing, which 
undoubtedly is and will be important. However, as researchers note, the personality-oriented 
education-communication model in no way ignores the systematic education and training of 
children, it does not disturb the consistency of the goals of pedagogical activity (Juodaityte, 
2002, p. 32). Basically, all modern searches in education are expressed by three postulates. 


Postulate 1: A child is an active subject with individual features, talents, and aspirations 
unique to him (axiom) — it is a personality, discovering and creating the culture of society, 
living in that society independently and forming himself independently, and becoming a full- 
fledged and free member of the nation (theorem) — it is important to educate the ability of 
young generation not only to live in a constantly changing world and environment, but also 
to change living conditions, to improve them, not being satisfied with what has been obtained 
from previous generations (didactic goals and tasks). 
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Postulate 2: The socialisation of the personality is realised in a polycultural society, 
characterised by diversity and interdependence (axiom) — the classical concept of culture 
as the process of transmission of values and beliefs from generation to generation through 
language and other symbol systems is inadequate and insufficient for present and future life 
(theorem) — modern culture is a set of interacting cultures, the space of human existence, 
for the creation of which it is necessary to train the young generation (in the construction of 
education goals, content, process, etc.) (didactic goals and tasks). 


Postulate 3: The most important and primary source of modern education (teaching and 
upbringing) is real life and direct experience (axiom) — education is effective when it is 
maximally moved nearer (or fully adequate) to modern life, is active and realised through 
cooperation (theorem) — emphasis is placed on active teaching, work in various groups 
(teams), cooperation in various project and innovative activities (didactic goals and tasks). 


Thus, it can be said that the axiomatics of education is a significant instrument for the 
cognition of educational phenomena, processes, and perspectives. Future education can no 
longer be based on postulates established by directives. It is no doubt that it is useful for educators 
to constantly remember the axiomatic statement — a mature thought gives birth to a wise action. 
Education is capable of changing society, just as society is capable of changing education. 
A fundamental open question is how to transform society through education. Of course, in a 
positive way. Finally, what changes in education can we expect in the next 20-30 years? Will 
a revolution take place in education? Predictions say that in the future even in the field of 
education, robots and machines will work instead of people. Does this mean that educational 
processes will take place without values and with very little interaction between educators 
and students. According to Eleyyan (2021), it is important to effectively implement significant 
changes in teaching/study programmes, learning/study environment, teacher — student roles and 
other changes, in order to cope with the challenges caused by the industrial revolution (4.0). 
Axiomatic theory in a general sense, or educational axiomatics, has real predictive power for 
education systems. This is undoubtedly important in education, where “trial and error” can turn 
into “trial by fire” for those being educated (Thompson et al., 2009). 
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Abstract 


The concept of hybridisation is one of the most difficult concepts for chemistry students to grasp at all 
levels of learning. Research showed the students conceptual difficulty ranged from their lack of the pre- 
requisite knowledge for grasping the topic hybridisation to chemical bond formation and orientations 
of atomic orbitals. This study investigated the difficulties Senior High School Students face in learning 
hybridisation. The study adopted a mixed-method approach using the sequential exploratory design. 
Purposive sampling was used to select six schools that offer elective chemistry subject. Simple random 
sampling was then used to select 120 Senior High School form I students to take part in the study. 
Convenient sampling was used to interview 24 students from the sample. Hybridisation Achievement 
Test (HAT) and Semi-structured interviews were self-constructed and used to collect data. Descriptive 
statistics and Content analysis were used to analyse the data. Results showed that, majority of students 
had difficulties in explaining the concept of hybrid orbitals, writing the electron configuration of ,C, 
explaining the effect of hybridisation on formation of chemical bonds in ,C, and type of hybridisation and 
shapes of compounds such as NH, OF, BCl,, CO, SiO, C,H, BeF, and CH, The study also showed 
that students had difficulties in explaining and demonstrating the formation of C = C double bond in 
ethene and other compounds. Equally students were challenged in demonstrating the differences between 
the formation of sigma and pi bonds in compounds. Students had difficulties in using electron orbital 
diagrams to explain the shape of CO, as linear. The study concluded that Senior High School form I 
chemistry students in the Upper West Region of Ghana lacked the most basic and fundamental concept of 
hybridisation. Teachers need to use conceptual change instructional approaches to teach hybridisation 
in order to foster students ’understanding and reduce misconceptions. 

Keywords: atomic orbitals, hybrid orbitals, hybridisation, mixed- method approach, sigma and pi-bonds 


Introduction 


The concept of hybridisation was proposed by Linus Pauling in 1931 to explain the 
rearrangement of the energy of individual atomic orbitals to produce new orbitals of equivalent 
energy. These newly formed orbitals, (hybrid orbitals), resulting from the mixing of other same or 
similar orbitals of different energies and shapes from other similar or same atoms form chemical 
bonds (Gillespie, 2004). The chemical bonds of compounds formed are either sigma or pi-bonds 
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(Gillespie, 2004). The newly hybridised orbitals possess the geometry and bonding properties of 
the newly formed molecules of chemical compounds (Petrucci, 2007). The new hybrid orbitals 
formed have some of the properties of the different atomic orbitals which go into forming them 
(Ameyibor & Wiredu, 1993). Bempong et al., (2009) described hybridisation as the processes 
of mixing two or more different atomic orbitals of different energies and shapes of the same or 
different atoms to form new hybrid orbitals which are equivalent in energies and shape. During 
the mixing of atomic orbitals to form new hybrid orbitals, electrons of the interacting atoms 
are shared between atomic orbitals and paired in overlapping orbitals of the atoms. In this inter 
atomic orbital mixing, the shared pair of electrons, share and occupy the same region of space. 
The orbital overlapping interactions of the atoms balance inter electronic and nuclear repulsions 
of the interacting atoms. This then results in formation of most stable orbital configurations 
(Essah & Emmanuel, 2019). 

Yet, Senior High School or Upper Secondary School Chemistry students, must have a firm 
grasp of the concept of hybridisation in order to predict molecular and electrical properties of 
substances (Essah & Emmanuel, 2019). In Ghana, the Senior High School chemistry syllabus 
emphasises the need for students to learn how to use models to help them explain molecular 
shapes and structures, chemical bonds, and their properties (Ministry of Education [MOE], 
2010). This aim of the Ministry of Education, (2010) is not different from other African 
countries. 


Research Problem 


Recent studies showed that, majority of chemistry students have difficulty understanding 
and assimilating chemical topics (Essumang & Bentum, 2012). Many of the research studies 
showed that most Senior High School or Upper Secondary School Chemistry students frequently 
confuse orbitals, shells, and orbital mixing and are unable to distinguish between atomic and 
molecular orbitals (Calis, 2018; Hanson et al., 2012; Koomson et al., 2020), and are therefore 
unable to comprehend the concept of hybridisation. Some specific students’ misconceptions of 
chemical topics include but not limited to bond angles, shapes of molecules, and their difficulty 
in understanding concepts of isotopy and allotropy (Schmidt et al., 2003). 

Many factors that contribute to students’ misconceptions of chemical topics and their 
subsequent poor academic achievement, including inadequate preparation by teachers, lead to 
poor instructional techniques employed by chemistry teachers (Okebukola, 2005); the inability 
of students to understand basic chemistry concepts; teachers inability to explain the sub- 
microscopic, macroscopic and symbolic learning (Taber, 2001); and the inability of learners 
to understand basic concepts in chemistry (Taber, 2001). Yet other studies found that students 
have trouble recognising chemical bonds (Demirci et al., 2016; Harrison & Treaguest, 2000; 
Kabapinar & Adik, 2006; Sen, & Yilmaz, 2013) and distinguishing chemical bonds from each 
other (Adik, 2003; Unal et al., 2001). 

Koomson et al., (2020) found that, the hybridisation state of central atoms in molecules 
was a challenge for students to determine. They also reported that students were perplexed by 
the angles of the bonds in BCI, and C,H, chemical compounds and couldn't relate the shape of 
the molecular species. Other research studies reported that the concept of hybridisation and 
atomic orbitals is one of the most difficult concepts for students to grasp at all levels of learning 
(Calis, 2018; Jian, 2014; Salah & Dumon, 2014). Calis (2018) reported that majority of students 
struggled to use atomic and hybrid orbitals to explain the formation of sigma and pi bonds 
in ethane, ethene and ethyne. Again, Calis (2018) found that students could not accurately 
demonstrate how the sp, sp”, and sp? hybrid orbitals are formed. 

The research therefore investigated into Senior High School students’ difficulties in 
learning hybridisation in chemistry, especially in the Upper West Region of Ghana. This region 
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is one of the regions in Ghana, in which Senior High School students performed abysmally in 
West African Senior High Schools’ Examinations. 


Research Question 


This study sought to answer the question: What are the Senior High School students’ 
learning difficulties in hybridisation in chemistry? 


Research Methodology 
General Background 


The study adopted a mixed- method approach using the sequential exploratory mixed- 
method design. The mixed-method approach enabled the study to gather, analyse, and integrate 
qualitative and quantitative data of the research (Creswell, 2014). The philosophy that 
underpinned the study is the pragmatist paradigm. 


Sample Procedure 


The study targeted Senior High School first year chemistry students in the Upper West 
Region, Ghana. A multistage sampling technique was used to sample 120 Senior High School 
first year chemistry students for the study. 

The first criterion used in the selection of the schools was Senior High Schools which 
offer chemistry as an elective subject. Twenty (20) Senior High Schools that offer chemistry as 
an elective subject were purposively sampled out of 32 Senior High Schools in the Upper West 
Region of Ghana. The Purposive sampling procedure was used because some of the Senior 
High Schools do not offer chemistry as an elective subject in their programme of instruction. 

The second criterion of the multistage sampling technique was to select schools which 
were category A Senior High Schools in the classification of Ghana Education Service category, 
and six schools were purposively selected. 

The third criterion of the multistage sampling technique was to select intact classes and 
six intact classes were simple randomly selected from each of the six category A Senior High 
Schools. At the fourth stage, twenty-four (24) students comprising of four (4) students each 
from the six intact classes were conveniently selected to take part in the interview. The selection 
was based on their performance in the hybridisation achievement test and included those who 
performed very well and those who performed poorly. 


Research Instruments 


Two instruments were used to collect data: Hybridisation Achievement Test (HAT) and 
Semi-structured interviews. 


Hybridisation Achievement Test (HAT) 


The Hybridisation Achievement Test (HAT) consisted of 15 open-ended questions on 
the concept of hybridisation. The questions were developed to cover areas such as concept of 
hybridisation, types and formation of hybrid orbitals and shapes of molecular compounds and 
formation of sigma and pi bonds. These topics were outsourced in the chemistry curriculum 
in Ghana (MOE, 2010). The questions were self-constructed to reflect concepts of the topic 
hybridisation in the chemistry curriculum. This was undertaken to identify major concepts 
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students are supposed to learn. The items were developed based on the profile dimensions 
outlined in the curriculum, based on the Bloom’s taxonomy of learning domains. Works of 
other researchers from literature on hybridisation also guided in the development of the items 
(Calis, 2018; Hanson et al., 2012; Koomson et al., 2020). The items were given to experts such 
as chemistry teachers and lecturers to make inputs and ensure content and face validity. 

A tubric for scoring the test was developed and students’ responses were put into three 
categories: Correct Response (CR), Wrong Response (WR) and No Response (NR). A student 
was categorised under correct response (CR) if all or most of his/her responses to an item were 
correct. A student was categorised under wrong response (WR) if most of his/her responses to 
an item were wrong. Students who did not attempt at all to the items were categorised as no 
response (NR). 

The Hybridisation Achievement Test was piloted with 40 SHS students. This was done to 
determine the reliability and then to ensure that the items were without ambiguity and students 
understood the items. The Kuder-Richardson formula 21 (KR-21) was used to determine the 
reliability of the test. The reliability was found to be 0.97 suggesting that the test was reliable. 


Semi-Structured Interview 


The Semi-structured interview items were self-constructed. To ensure trustworthiness of 
the interview data, the interview protocol was given to experts to make inputs and pilot tested. 
The interview transcripts were read to the students to ensure that the responses were exactly 
what they intended to say. Twenty- four (24) students from the sample were interviewed. 


Data Collection Procedure 


Permission was sought from the Senior High Schools’ administration using an 
introductory letter from the Science Education Department. A Pre-Test was conducted using 
the Hybridisation Achievement Test (HAT) to unearth the learning difficulties students have 
in hybridisation. The students were taught the concept of hybridisation by their teachers for a 
period of four weeks. 

Post-Test was conducted after the instruction using the Hybridisation Achievement Test 
(HAT) to identify the learning difficulties of students. The items on the Pre-test and Post-test 
were similar. 

Semi-structured interview was conducted to further identify students learning difficulties 
in hybridisation and to validate the responses from the Hybridisation Achievement Test (HAT). 
The interview was face to face and lasted between 30-60 minutes. The interview was recorded 
and transcribed verbatim. 


Data Analysis 


Data was analysed using Statistical Package for Social Sciences (SPSS) version 25. 
Descriptive statistics such as frequency was used to analyse the data. The frequencies were 
used to determine the number of students who gave Correct Responses, Wrong Responses or 
No responses to the items of the test. 

Content analysis was used to analyse the interview data. Content analysis is an approach 
to quantify qualitative information by systematically sorting and comparing items of information 
in order to summarize them. Often this process entails turning a large set of raw data into 
useable evidence through data reduction methods (Hawkins, 2013). The data was coded and 
categories into themes were based on whether students’ responses were correct or wrong. The 
results were presented in tables and figures. 
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Research Results 


Learning difficulties students encounter in hybridisation 


Difficulties in the Concept of Hybridisation 


Table 1 shows the difficulties students have in the concept of hybridisation. These 
difficulties were revealed in their responses to items in the Hybridisation Achievement Test. 


Table 1 
Students’ Learning Difficulties on the Concept of Hybridisation 


Learning difficulties Category Pre-test Post-test 
f % f % 

NR 19 15.8 1 0.8 

Explaining the term hybridisation WR 55 45.8 45 37.5 
CR 46 38.3 74 61.7 

NR 45 37.5 12 10.0 

Explaining the term Hybrid orbitals WR 67 55.8 87 72.5 
CR 8 6.7 21 17.5 

NR 46 38.3 24 20.0 

Stating the effect of hybridisation on bonds. WR 54 45.0 36 30.0 
CR 20 16.7 60 50.0 

NR 24 20.0 3 2.5 
te the ground state Electron Configuration WR 50 a7 50 A7 
CR 46 38.3 67 55.8 

7 NR 48 40.0 3 2.5 
a6. the excited state Electron Configuration WR 56 46.7 a 34.2 
CR 16 13.3 76 63.3 

NR 35 29.2 8 6.7 

Stating the number of hybrid orbitals in ,C. WR 62 51.7 77 64.2 
CR 23 19.2 35 29.2 


(From researchers’ data, 2021). 
NR — No Response, WR — Wrong Response, CR — Correct Response 


The pre-test results revealed that, many of the students gave wrong responses or did not 
respond to the items on the concept of hybridisation, hybrid orbitals, effect of hybridisation on 
bonds, ground state and excited state configuration of carbon and stating the number of hybrid 
orbitals in carbon. This implies that, before the instruction, students had difficulties in the 
concept of hybridisation. 

From the post-test results, 74 of the students representing 61.7% correctly explained 
the term hybridisation. Also, 45 of the students representing 37.7% gave wrong explanations 
to the concept of hybridisation while | student representing 0.8% did not make an attempt in 
explaining the term hybridisation. This implies that, students had less difficulty in explaining 
the term hybridisation. 
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The post-test results also revealed that, 21 of the students representing 17.5% correctly 
explained the term hybrid orbitals while 87 students representing 72.5% gave wrong 
explanations. Also, 12 students representing 10.0% did not respond. This implies that students 
had difficulty in explaining the term hybrid orbitals. 

Results from the interview data also revealed that, students had difficulty in learning 
the concept of hybridisation. For example, regarding the concept of hybridisation, this is what 
a student said. 


Researcher: The hybrid orbitals formed after hybridisation have a higher energy than the 
individual orbitals mixed. Explain your answer. 
Student 1: (Correct response): ‘’The orbitals mixed always have a higher energy which makes the 
hybrid orbitals formed to have a lower energy”. 


Student 2 (wrong response): ‘’The individual orbitals are strong and stable when mixing them due 


to the type of hybridisation ”. 


Student 3 (wrong response): ‘’Hybrid orbital is the central atom in which hybridisation is formed 
with the same energy and shape”. 


Student 4 (wrong response): “Individual orbitals have low energy and when mixed will have 
orbitals with increasing energy ”’. 


Also, students had difficulty in defining the concept of hybridisation. This was evident in 
some of their definitions as follows: 


Researcher: Explain the term hybridisation. 

Student 20 (wrong response): “’hybridisation is a mixing of orbital to form the same orbital”, 
Student 15 (correct response): “hybridisation is the mixing up of two or more different atoms of 
the same energy and shape to form a new hybrid orbital of the same energy and shape”, 

Student 8 (wrong response): “’ hybridisation is the mixing of two or more atomic orbitals of 
different energy and shape to form a new bond” 


Many of the students also had difficulty in explaining the term hybrid orbitals. This is 
what they had to say: 


Researcher: Explain the term hybrid orbitals. 

Student 3 (correct response): They are orbitals formed when two or more orbitals of different 
energies and shapes mix to form new orbitals of equivalent energy and shape 

Student 12 (wrong response): ‘these are point use different the two or more - bond energy all 
together”, 

Student 8 (wrong response): ‘’is defined as the covalent molecular atom as the result of the head- 
on overlap of orbitals”’, ‘ 

Student 18 (wrong response): ‘is the mixing of two or more orbitals when new orbitals are formed 


in different energy and shape”. 


Students also could not state the effect that hybridisation has on bonds. Example of their 
responses are seen in the excerpts below: 


Researcher: what is the effect of hybridization on bonds? 
Student 5: ‘’ Hybrid orbitals predict equal energy and shape” 
Student 4: ‘’ The hybridisation has ground state”, 

Student 11: ‘’ The outermost electrons take part in the bonding” 
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636 Students had difficulty in writing the ground and excited-state electron configuration of 


<C. For example, below are some of the student’s responses: 


Researcher: write the ground and excited state electron configuration of ,C. 
Student 3: s/, 57; 8’, s*; spy spx spz 

Student 5: Js? 2s? 2px’; 1s! 2s! 2px! 2py! 2pz! 

Student 16: ’/s? spx? spy’; s? spx? spy? 


Again, many of the students were not able to state the number of hybrid orbitals in .C. 
The excerpts below are some of their responses: 


Student | (wrong response): 4 

Student 9 (wrong response): four sp? 

Student 2 (wrong response): sp? 

Student17 (wrong response): one type of hybrid thus sp 


Formation of Sigma and Pi bonds 
Table 2 shows the difficulties students had in the concept of formation of sigma and pi 
bonds. These difficulties are revealed in their responses to items in the hybridisation achievement 


test. 


Table 2 
Learning Difficulty of Students on the Formation of Sigma and Pi Bonds 


Pre-test Post-test 
Learning Difficulty Response 
% f % 
NR 103 85.8 47 39.2 
Formation of C = C double bond in ethene. WR 17 14.2 70 58.3 
CR - - 3 2.5 
NR 81 67.5 15 12.5 
Formation of pi bond and sigma bond using an electronic WR 38 317 81 675 
diagram. 
CR 1 0.8 24 20.0 
NR 88 73.3 28 23.3 
Formation of sigma and pi bonds in Ethyne (H - C= C - H). NE = ice a tA 
CR - - 6 a 


(From researchers’ data, 2021). 
NR -No Response, WR - Wrong Response, CR — Correct Response 


The pre-test results revealed that, many of the students gave wrong responses or did not 
respond to the items on the formation of C = C double bond in ethene, formation of pi bond 
and sigma bond using an electronic diagram and the formation of sigma and pi bonds in ethyne 
(H - C=C-H). This implies that, the concept of formation of sigma and pi bonds before the 
instruction, was difficult for students. 

The post-test results revealed that, only 3 students (2.5%) of the students correctly used 
an appropriate diagram to explain the formation of C = C electron double bond in ethene while 
70 students (58.3%) gave wrong explanations, and 47 students (39.2%) did not respond. The 
high percentage implies that, students had difficulty in explaining the formation of C = C double 
bond in ethene. 
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Results from the interview data also revealed that, students had difficulty in learning the 
concept of formation of sigma and pi bonds in ethene. Some students’ responses are presented 
in Figure | below. That is, the majority of students had a difficulty in drawing the trigonal planar 
shape of Carbon and indicating the unhybridized 2p, orbital for the formation of sigma and pi 
bonds in ethene. 


Figure 1 
Students’ diagrammatic representation of the C = C bond in an alkene 


In addition, the post-test results also revealed that, 24 students (20.0%) of students 
correctly used the appropriate electron diagram to differentiate between the formation of pi 
bond and sigma bond while 81 students (67.5%) gave wrong explanations, and 15 students 
(12.5%) did not respond. This implies that, students had difficulty in explaining the formation 
of pi and sigma bonds in ethene. 

Also, students could not use electronic diagrams to differentiate between pi and sigma 
bonds. The majority of students had difficulty in using s, p, and d hybrid orbitals to differentiate 
between the formation of pi and sigma bonds. Figure 2 shows a sample of students’ drawings. 


Figure 2 
Students’ Drawings of pi and sigma Bonds 
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However, some 6 students (5%) correctly used orbital diagram to describe the formation 
of sigma and pi bonds in ethyne (H — C = C— H), while 86 students (71.7%) could not describe 
the bonding in ethyne using orbital diagrams. Moreover, some 28 students (23.3%) did not 
respond at all. This implies that, majority of students had difficulty in using diagrams to describe 
the formation of sigma and pi bonds in ethyne. Figure 3 shows samples of students’ drawings. 
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Figure 3 
Students’ Drawings of the Formation of sigma and pi Bonds in Ethyne 
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Some other students gave explanations of the formation of sigma and pi bonds without 
illustrating with orbital diagrams. However, these explanations were wrong. Figure 4 shows 
samples of students’ explanations. 


Figure 4 
Students’ Explanation of the Formation of pi and sigma Bonds 
1) S wrbeayiee, it we A bend Feemed by Mee bieeuk ee or wes re 4 
Weed to hia 


i > bid be nid « jx G@ bo. ref forrted b J Ny» Leder ca/ Ae bn eng 
iat @ way oVerlar at citeont:, orks ites of (e eborrs af 
Infern ac leas xsl 


as ‘your guide, 


Sty nia bw tsa he ror ret fy dupa 9 Y Minty 
(head te heranJd 2? oVverl a;77 ov Jord thr in Ler 4 

2 &L the la Jere’ or SY ove as 
pre& bound *s © brevd Ferxvevrd & : m 
Aerio orbvfes grtertaya hoerg & dupeohion of rrre 
Calar (whernal leaner aver's - u%e & brdazmrm fb eng tie— 


Furthermore, students had difficulty in explaining that a triple bond consists of one sigma 
bond and two pi bonds. This is what some students had to say: 


Researcher: Explain the type and number of bonds in ethyne. 

Student 1: ethyne have two pi bonds and one sigma bond 

Student 16: you can use carbon to explain all the hybridisation — sp, sp’ and sp* 
Student 9: the one sigma and one pi bonds have a small degree of overlap 
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Students could not also explain the number of sigma and pi bonds in CO molecule. 


Researcher: How many sigma and pi bonds are in CO? Explain your answer. 
Student 12: CO has I sigma and I pi bond because in the excited state is only one electron that is 


used in the bonding and the one-electron is the sigma bond and the other one is the pi bond 
Student 2: CO has one sigma, no pi bond...the shape is linear 


Student 18: CO has one sigma and one pi bond...this is because it is having two lone pair of 


electrons 


Student 14: CO has six...because sigma bonds are two and pi bonds are four 


Types and Formation of Hybrid Orbitals 


Table 3 shows the results of students’ difficulties in the types and formation of hybrid 


orbitals. 


Table 3 


Learning Difficulties of Students on the Types and Formation of Hybrid Orbitals 


Learning difficulty Response = _ 
% F % 
NR 83 69.2 32 26.7 
Describe the formation of sp* hybrid orbitals in CCl,. WR 37 30.8 80 66.7 
CR 8 6.7 
NR 73 60.8 7 5.8 
Shape and bond angles of sp hybrid orbitals. WR 26 21.7 29 24.2 
CR 21 17.5 84 70.0 
NR 98 81.7 29 24.2 
Type of hybridisation of central atom in NH, WR 19 15.8 42 35.0 
CR 3 25 49 40.8 
NR 100 83.3 30 25.0 
Type of hybridisation of central atoms in OF, WR 20 16.7 47 39.2 
CR 43 35.8 
NR 102 85.0 30 25.0 
Type of hybridisation of central atoms in BCI, WR 17 14.2 57 47.5 
CR 1 08 33 27.5 
NR 99 82.5 32 26.7 
Deduction of shape of NH, WR 21 17.5 73 60.8 
CR 15 12.5 
NR 101 84.2 32 26.7 
Deduction of the shape of OF, WR 19 15.8 73 60.8 
CR 15 12.5 
NR 102 85.0 37 30.8 
Deduction of the shape of BCI, WR 17 14.2 64 53.3 
CR 1 08 19 15.8 
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NR 82 68.3 10 8.3 
Type of hybridisation of central atom in CO, WR 28 23.3 40 33.3 
CR 10 83 70 58.3 
NR 85 708 11 9.2 
Type of hybridisation of central atom in SiO, WR 35 29.2 78 65.0 
CR 31 25.8 
NR 93 £5. 13 10.8 
Type of hybridisation of central atom C,H, WR 26 21.7 70 58.3 
CR 1 08 37 30.8 
NR 95 79.2 16 13.3 
Type of hybridisation of central atom in BeF , WR 20 16.7 60 50.0 
CR 5 42 44 36.7 
NR 95 79.2 17 14.2 
Type of hybridisation of central atom in C,H, WR 22 18.3 76 63.3 
CR 3 2.5 27 22.5 
NR 82 683 22 18.3 
Shape of CO, WR 31 25.8 44 36.7 
CR rg 5.8 54 45.0 
NR 89 74.2 = 24 20.0 
Shape of SiO, WR 31 25.8 80 66.7 
CR 0 0.0 16 13.3 
NR 94 78.3 = 23 19.2 
Shape of C,H, WR 25 208 64 53.3 
CR 1 08 33 27.5 
NR 94 78.3 27 22.5 
Shape of BeF , WR 23 19.2 56 46.7 
CR 3 2.5 37 30.8 
NR 96 80.0 26 21.7 
Shape of C,H, WR 24 20.0 78 65.0 
CR - - 16 13.3 


(From researchers’ data, 2021). 
NR-— No Response, WR — Wrong Response, CR — Correct Response 


The pre-test results revealed that, many of the students gave wrong responses or did not 
respond to the items on the formation of sp* hybrid orbitals in CCl ,, Shape and bond angle of sp 
hybrid orbitals, the type of hybridisation of the central atoms in and shape of NH,, OF, BCI, 
CO,, SiO,, C,H,, BeF, and C,H, molecules. This implies that, before the instruction, students 
had difficulties in the concept of types and formation of hybrid orbitals. 

The post-test results revealed that, 8 students (6.7%) correctly explained the formation of 
sp* hybrid orbitals in CCl, while 80 students (66.7%) gave wrong explanations, and 32 students 
(26.7%) did not respond. The high percentages of wrong and no response implies that, students 
had difficulty in explaining the formation of sp’ hybrid orbitals in CCl, with diagrams. 

The majority of students could not use a diagram to explain the formation of sp* hybrid orbitals 
in CCl, given an s and three p orbitals. 
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the s and p orbitals to draw the tetrahedral shape of the sp’ hybrid orbital. Some samples of 


students’ responses are presented in Figure 5. 


Figure 5 
Students’ Responses on the Formation of sp? Hybrid Orbitals in CCI, 
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It was also revealed in the post-test that, 84 students representing 70.0% correctly 
used diagrams to explain the shape and bond angle of sp hybrid orbitals. Also, 29 students 
representing 24.2% gave wrong explanations while 7 students (5.8%) did not respond. This 
implies that most of the students had less difficulty in explaining the shape and bond angle of 
sp hybrid orbitals. 

Also, the post-test results showed that, 49 students (40.8%) correctly stated the type 
of hybridisation of the central atom in NH, while 42 students (35.0%) stated wrongly and 
29 students (24.2%) not answering. Also, 43 students (35.8%) correctly stated the type of 
hybridisation of central atom in OF, while 47 students (39.2%) gave wrong responses, and 30 
students (25.0%) did not respond. With the type of hybridisation of the central atom in BCl,, 
33 students (27.5%) gave correct response while 57 (47.5%) gave wrong responses, and 30 
students (25.0%) did not respond. This implies that, students had difficulties in stating the type 
of hybridisation of the central atoms in NH,, OF, and BCl.. 

Furthermore, post-test results revealed that, 15 students representing 12.5% deduced the 
shape of NH, correctly while 73 students (60.3%) provided wrong responses, and 32 students 
(26.7%) did not respond. Also, with the shape of OF,, 15 students (12.5%) correctly deduced 
the shape while 64 students (53.3%) gave wrong response, and 32 students (26.7%) did not 
respond. For BCI, molecule, 19 students (15.8%) deduced the shape correctly while 64 students 
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(53.3%) gave wrong responses, and 37 students (30.8%) provided no response. This implies 
that, students had difficulty in deducing the shape of NH,, OF, and BCl.. 

The results further revealed that, 70 (58.3%), 31 (25.8%), 37 (30.8%), 44 (36.7%) and 27 
(22.5%) students provided correct responses on the type of hybridisation of the central atoms 
in CO,, SiO,, C,H,, BeF, and C,H, respectively. Also, 40 (33.3%), 78 (65.0%), 70 (58.3%), 
60 (50.0%) and 76 (63.3%) students gave wrong responses on the type of hybridisation of the 
central atoms in CO,, SiO, C,H,, BeF, and C,H, respectively. 

The post-test results also revealed that, 54 (45.0%), 16 (13.3%), 33 (27.5%), 37 (30.8%) 
and 16 (13.3%) of students stated correctly the shapes of CO,, SiO,, C,H,, BeF, and C,H, 
respectively. Also, 44 (36.7%), 80 (66.7%), 64 (53.3%), 56 (46.7%) and 78 (65.0%) gave wrong 
responses on the shapes of CO,, SiO,, C,H,, BeF, and C,H, respectively. 

Interview responses from students also revealed students’ difficulties in the concept 


types and formation of hybrid orbitals. 


Researcher: The central atom C in CH, is sp’ hybridised. Explain your answer. 
Student 3: "the core atom of CH, is sp orbital"; 

Student 16: "because C in CH, has two lone pair of electrons and is bent in the form"; 
Student 7: "because the lone pair of electrons on C in CH, will move them down" 


Also, on the type of hybridisation on the carbon atoms in ethyne, students had this to say. 


Researcher: A// the carbon atoms in ethyne are sp’ hybridised. Explain your answer. 

Student 1: ‘In Ethyne there is one pi bond and two sigma bonds which join together to form sp 
in carbon”. 

Student 2: Because all the carbon atoms are having one angle of 109.5 and a shape of trigonal” 
Student 3:” You can use carbon to explain all the hybridisation — sp, sp* and sp?” 


3 


Shapes of Molecular Compounds 


Table 4 shows the difficulties students have in shapes of molecular compounds. These 
difficulties are revealed in their responses to items in the Hybridisation Achievement Test. 


Table 4 
Students’ Learning Difficulties in Shapes of Molecular Compounds 


Pre-test Post-test 
Learning difficulty Response 

% F. % 
Explain why H,0 is not a linear molecule using hybridisation. ue 59 a2 19 158 
WR 59 49.2 50 41.7 
CR 2 Af 51 42.5 
Explain why CO, is a linear molecule using hybridisation. NR 69 57.6 v 14.2 
WR 51 42.5 50 41.7 
CR - - 53 44.2 
Explain why NH, and H,O have bond angles of 107° and NR 92 16.7 40 33.3 
104.5° respectively. WR 28 23.3 70 58.3 
CR - - 10 8.3 
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_ NR 92 76.7 17 14.2 
Draw an orbital overlap diagram to represent the bonding in WR 28 33 95 792 
the methyl (CH,) group. 
CR - - 8 6.7 
NR 63 62.5 9 15 
Matching sp, sp? and sp’ hybrid orbitals with shapes of WR 2 47 25 20.8 
molecules and compounds. CR 31 25.8 86 717 
With the aid of electron bonded diagrams show the lone pair NR 93 17.5 63 62.5 
of electrons and bond pair of electrons present around Al and WR 7 225 6 46.7 
P in AICI, and PCI, molecules? : 
CR - - 1 0.8 
With the help of a diagram show the electron bonding in CO NR 10 90.0 56 46.7 
molecule. WR 12 10.0 64 53.3 


(From researchers’ data, 2021). 
NR -— No Response, WR — Wrong Response, CR — Correct Response 


The post-test results showed that, 51 students (42.5%) correctly explained why H,O is 
not a linear molecule while 50 students (41.7%) explained wrongly, and 19 students (15.8%) 
did not respond. This implies that, students had difficulty in using hybridisation to explain why 
H,0O is not a linear molecule. 

Also, 53 students (44.2%) used hybridisation to explain correctly that, CO, is a linear 
molecule while 50 students (41.7%) provided wrong responses, and 17 students (14.2%) did 
not respond. This also implies that students had difficulty in using hybridisation to explain that 
CO, is a linear molecule. 

The post-test results further revealed that, 10 students (8.3%) correctly accounted for 
the difference in bond angles between NH, and H,O using diagrams while 70 students (58.3%) 
gave wrong explanations, and 40 students (33.3%) did not respond. This implies that, students 
had difficulty in using diagrams to account for the difference between NH, and H,O with bond 
angles of 107° and 104° respectively. 

In addition, the post-test results indicated that, 8 students (6.7%) correctly used orbital 
overlap diagrams to represent the bonding in methyl (CH,), while 95 students representing 
79.2% gave wrong responses, and 17 students (14.2%) did not respond. This implies that, 
majority of the students had difficulty in using orbital overlap diagrams to represent the bonding 
in methyl (CH,). 

The results from the post-test show that, none of the students could use a diagram to 
show the bonding in CO molecule while 64 students (53.3%) gave wrong response, and 56 
students (46.7%) did not respond. This implies that, students had a difficulty in using diagrams 
to show the bonding in CO molecule. 

The results revealed that majority of students were not able to use diagrams to show 
the arrangement of bonding electrons in BeCl, and BCl,. Figure 6 shows samples of students’ 
responses. 
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Students Drawings of the Bonding in BeCl, and BCI, 
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Students could not account for the difference in bond angles in NH, and H,O. Figure 7 
shows samples of students’ responses. 


Figure 7 
Students’ Responses on Bond Angles of NH, and H,O 
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In addition, students could not draw an orbital overlap diagram to represent the bonding 
in methyl (CH,). Majority of students failed to understand that the central atoms in CH, and 
CH, are sp’ with the only difference being the number of Hydrogen atoms in the tetrahedral 
structure. The difficulty of students is seen in some of their responses as shown in Figure 8. 
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Students’ Drawings on Bonding in Methyl (CH,) 
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Furthermore, students were not able to use orbital diagrams to show the number of lone 
pair and bond pair electrons of Al and P in AICI, and PCl, molecules. Majority of the students 
could not draw the orbital diagrams and identify the bond pair and lone pair electrons in AICI, 
and PCl.. 
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Students’ Drawings of Bonding in AICI, and PCI, Molecules 
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Students had difficulty in showing the bonding in CO. Some of the responses from 
students are shown in Figure 10. 


Figure 10 
Students’ Drawings on the Bonding in CO Molecule 
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15. With the help of a diagram show the electron bonding in CO molecule, 
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15. With the help of a diagram show the electron bonding in CO molecule 
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Discussion 


The results revealed that, 61.7% of the students were able to correctly define the concept 
of hybridisation while 37.5% gave incorrect definitions. The findings are consistent with that 
of Calis (2018) who reported that the number of students who correctly explain the definition 
of the hybridisation concept with their own census is 35%. Calis (2018) identified that 20% 
of the students considered the bonds made by the carbon atom as hybridisation, while 25% 
of the students define hybridisation as bringing orbitals in different energy levels to the same 
energy level. The findings supported that the misconceptions in the definitions reported by Calis 
(2018) since 37.5% of students could not define hybridisation. In a study about hybridisation 
and atomic orbitals, Nakiboglu (2003) stated that the students could not identify hybrid types 
or define the hybridisation concept. In his study, Zoller (1990) stated that the students confused 
the hybridisation concept with electronic orbitals and could not really define it. Taber (2001) 
revealed that the students thought that electrons in hybrid orbitals were the s, p, d atomic orbitals 
and they stated that hybrid orbitals were similar to atomic orbitals. 

Fifty percent (50%) of students stated correctly the effect of hybridisation on bonds 
while 30% gave wrong responses. Many of the students (55.8%) were able to write the ground 
state configuration of carbon, while 41.2% of them could not. Also, majority of the students 
(63.3%) were able to write the excited state configuration of carbon, while 34.2% of them could 
not. Majority of the students (64.2%) could not state the number of hybrid orbitals of carbon. 

However, Hanson et al., (2012) in their study found that more than half of the participants 
did not understand the meaning of the term, hybridisation. They reported that 72 of the 
participants (93.2%) failed to respond correctly to the set question on the hybridisation or did 
not answer it at all. The study also found that 72.5% of students gave wrong definitions of 
hybrid orbitals. This finding agrees with the work of Hanson et al., (2012) who reported that 
students had a better conceptual understanding of the concept of atomic orbitals. 

The majority of students had difficulty in stating the type of hybridisation in and shapes 
of NH,, OF,, BCI,,CO,, Si0,, C,H,, BeF,, and C,H,. Koomson et al., (2020) also found that, 
students struggled to identify the hybridisation of central atoms in molecules. Students could not 
also state the bond angle of BCI, and C,H,, and they could not relate bond angles to the shape 
of molecules. They reported that some students struggled to identify the hybridisation state of 
central atoms in molecules, that others were perplexed by the bond angles of BCI, and C,H, and 
couldn't relate the shape of the molecular species, that students couldn't define hybridisation, 
that students had a poor understanding of hybridisation’s contributing effect on bonds, and that 
student’s diagrammatic poor representation of why water is not a linear molecule. 

Again, students could not explain why the central atom in CH, is sp’ hybridised. 
Furthermore, many of the students failed to explain correctly why all the carbon atoms in 
ethene are sp” hybridised and why the carbon atoms in ethyne are sp hybridised. Majority of the 
students could not explain the bond angles of molecules such as NH, and H,O. Hanson et al., 
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(2012) reported that students experience conceptual difficulty as a result of understanding the 
meaning of s, p, d and f orbitals and how they are oriented in space. Again, the results confirmed 
the findings of 2. Nakiboglu (2003) and Hanson et al., (2012) that, pre-service teachers they 
used for their study displayed major misconceptions about atomic orbitals and hybridisation 
and students were unaware of the role hybridisation played in the formation of covalent bonds. 
Calis (2018) supported with his findings that, 75% of students could not accurately demonstrate 
how sp, sp* and sp* hybrid orbitals were formed while 25% could. 

Tilahun, and Tirfu (2016) reported that students had poor knowledge in the concept of 
hybridisation orbital like sp, sp”, and sp* which are the bases of chemical bonding. They did 
not recognize the importance of sp, sp? and sp? in determining single, double, and triple bonds 
respectively. Calis (2018) further found that 25% of the students were able to show the sp, sp” 
and sp* hybrid orbital formations correctly and recognize these orbitals. These students also 
did not specify how many identical hybrid orbits were formed, although they could show the 
formation of sp’, sp’ and sp hybrid orbital from the p orbital with s orbital. Also, most of the 
students did not show the hybrid orbital formation correctly. This result is also seen as a result 
of students' tendency to learn by memorizing the hybridisation phenomenon. 

The majority of students were not able to use diagrams to show the arrangement of 
bonding electrons in BeCl, and BCl,. Also, most of the students (57.5%) in the post test results 
had difficulty in explaining why H,O is not a linear molecule. This supports the findings of 
Koomson et al., (2020) that students could not draw and represent with diagrams explanations 
of why water is not a linear molecule. 

The results also showed that students could not correctly state the hybridisation of the 
central atoms in NH,, H,O, OF,, CO, and SiO,. This is consistent with other findings. For 
example, Uyulgan and Akkuzu (2016) in a study on conceptual understanding of the molecular 
structures of compounds found that 49.6% of students were not able to give the hybrid type of 
the central atom in the molecules while other students (31.6%) gave mostly incorrect answers, 
showing that the students had difficulty in identifying the hybrid type. Also, students had 
misconceptions about hybridisation. The students were not able to state that hybrid orbitals 
occur as a result of overlapping of orbitals. 


Conclusions 


In general, from the study most of the students had difficulty in understanding the 
concept of hybridisation. However, the concept of hybridisation is one of the key topics science 
students need to have adequate understanding to progress in the field of chemistry. This is 
because knowledge of hybridisation is pre-requisite to learning other topics in chemistry such 
as chemical bonding and organic chemistry. Unfortunately, research found that hybridisation 
is difficult to understand by most students at all levels of schooling. In order to find ways of 
minimizing the difficulties students encounter in learning hybridisation, the root cause as well 
as the specific areas of the concept students find to be more problematic must be known. 

It is therefore important to detect and take into consideration students’ prior conceptions 
when planning chemistry lessons. Understanding the concept of hybridisation requires the 
connection of different abstract concepts such as atomic orbitals, chemical bonding, and 
molecular compounds. This work found out the peculiar hindrances that impede senior high 
school science students understanding of the concept of hybridisation. 

The major areas of difficulties in learning hybridisation identified from the study included the 
inability of students to: 

e define the concept of hybridisation, 

e explain the types and formation of hybrid orbitals, 

e explain the formation of sigma and pi bonds in ethene and ethyne, 
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e draw electron orbital diagrams of some molecules, and 

e draw the shapes of molecules. 

There is the need for chemistry teachers to use student-cantered teaching approaches 
that will foster understanding of students in hybridisation. Again, teachers should use teaching 
and learning materials such as molecular/atomic models to teach hybridisation for conceptual 
understanding. In addition, conceptual methods and analogies should be used to teach shapes of 
hybrid orbitals and their orientations. Emphasis should be placed on the relationship the type of 
hybridisation of the central atom has on the molecular geometry of a molecule. For students to 
gain conceptual competency on the covalency of hybrid orbitals in bond formation, they must 
be introduced to more worked examples. 

It is therefore suggested that further research on using cutting-edge instructional 
technology or teaching learning materials should be conducted to minimize if not eliminate the 
learning difficulties associated with the findings of the research. 
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Abstract 


Teachers who teach English as a foreign language (EFL) should constantly develop their skills and 
knowledge by taking part in professional development programs (PDPs) which keep them aware of 
the most recent teaching trends. This mixed method study seeks to explore how PDPs influence EFL 
teachers’ teaching practices in activating 21" century skills, and therefore their students’ acquisition 
of these skills. It also examines the correlation between EFL teachers’ activation of 21" century skills 
and their students’ acquisition of these skills. The study used a sequential explanatory research design 
by utilizing questionnaires and semi-structured interviews. The results revealed that the two dimensions 
of 21" century skills that got the highest level of activation were the collaboration and communication 
skills. Furthermore, the results showed that the students acquire all 21" century skills with different 
levels of acquisition. Finally, the results showed that EFL teachers’ activation of 21" century skills had 
no significant direct correlation with students’ acquisition of these skills. These findings could provide 
insights on the area of PDPs that need to be researched more in different contexts. They also suggested 
that PDPs designers should incorporate all dimensions of 21" equally to be activated accurately by 
teachers in higher education institutions. 

Keywords: professional development, EFL teachers, EFL learners, 21" century skills activation, 21“ 
century skills acquisition 


Introduction 


There are many things that require changes over time which means that they are 
developing. Thus, teaching is a developmental process, and this is indicated by the changes 
of teachers’ skills, knowledge, attitudes, and behavior (Topkaya & Yelik, 2016). Therefore, all 
teachers either new or experienced need a wide variety of ongoing opportunities to improve their 
pedagogical knowledge and skills by attending professional development programs (PDPs). 
Thus, these programs for in-service teachers at higher education will help increase the quality 
of all teachers which will lead to the best results from their students (Compen et al., 2019). 

Professional development (PD) was defined by Darling-Hammond et al. (2017), as an 
organized professional learning that changes teacher knowledge and practices and enhances 
student learning outcomes. Merchie et al. (2018) also agreed with this definition and defined 
PD as activities that are designed for and presented to teachers in order to enhance their own 
and their students’ knowledge, skills, and attitudes. These definitions reflect important elements 
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that PD can influence, such as teachers’ knowledge, skills, and attitudes as well as students’ 
learning. 

As we are in the 21* century, teachers’ skills and practices must be aligned with enhancing 
21“ century skills at higher education, thus all PDPs are needed to help university teachers learn 
and improve their knowledge and instructions required to teach 21“ century skills (Darling- 
Hammond et al., 2017). Today’s young people must adapt to a quickly changing environment, 
which necessitates going beyond fundamental formulaic knowledge and abilities. Therefore, 
higher education policy should move from rote learning and fact memorizing towards the 
development of 21* century skills (Cruz, et al., 2021; Urbani et al., 2017). Andrade (2016) 
reported that teaching content development in higher education must be compatible with the 
acquisition of global skills from the 21* century, such as problem solving, critical thinking, oral 
and written communication, collaboration, and information literacy. 

In addition, in the field of English Language Teaching (ELT), the development of 
teaching language skills and its combination with language learners’ ever-changing needs such 
as 21“ century skills have become a main focus (Cinar, 2021). The 21* century skills at higher 
education help students to enter the work life with the capacity to think creatively and critically, 
interact with others, take initiative when faced with a problem, and fully utilize technology 
(Okolie et al., 2021; Weeks et al., 2020). Thus, learners in higher education need to be prepared 
to acquire the 21* century skills as their wants in this challenging era indicated that the teaching 
materials must be able to promote these skills (Menggo et al., 2019). In addition, Cinar (2021) 
claimed that every element of 21“ century skills becomes a requirement for English language 
teaching. Thus, administrators need to develop a plan to implement PDPs to support teachers 
with 21% century skills development in the classroom (Stegall, 2016). 


Research Aim and Research Questions 


The literature reveals that there are some research studies that have studied the impact 
of PDPs on students’ academic achievement such as Basma and Savage (2018); Miles (2019); 
and Warner, et al. (2019), however, the correlation between teachers’ participating in PDPs and 
students’ acquisition of 21“ century skills in higher education is not clear. Thus, teachers who 
are involved in PDPs are now challenged with the task of incorporating 21* century skills into 
their classrooms at higher education institutions. 

Although PDPs play a crucial role in changing higher education teachers’ practices and 
attitudes towards PDPs and subsequently students’ outcomes, few studies have focused on this 
topic particularly in Saudi Arabia. Hence, this mixed method study attempts to contribute to the 
knowledge base by exploring the correlation between EFL teachers’ practices in activating 21* 
century skills and their students’ acquisition of these skills. Therefore, this study was designed 
to answer the following research questions: 

1. To what extent does participation in PDPs at higher education help EFL teachers 

activate 21“ century skills? 

2. To what extent do learners of English course at higher education acquire 21* 

century skills? 

3. Is there a correlation between EFL teachers’ activation of 21* century skills and their 

students’ acquisition of these skills? 


Literature Review 
The Today’s young people must adapt to a quickly changing environment, which 
necessitates going beyond fundamental formulaic knowledge and abilities. Therefore, education 


policy should move from rote learning and fact memorizing towards the development of 21* 
century skills (Urbani et al., 2017). There are various interpretations of what 21‘ century 
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skills are. The Partnership for 21*' Century Learning Skills has established a structured vision 
for the overall educational process titled the “Framework for 21“ Century Learning”. It was 
first released in 2007 and then updated in 2016. This framework determines the skills which 
are crucial for students’ success in life and work. It contains a combination of various skills, 
competences along with knowledge and expertise in the field of academic disciplines. Four 
skill areas have been identified to outline the 21“ century students’ outcomes which are: (a) key 
subjects and 21‘ century themes; (b) life and career skills; (c) learning and innovation skills: the 
4Cs (i.e., critical thinking, communication, collaboration, and creativity); and (d) information, 
media, and technology skills (Battelle for Kids, 2021). 

Moreover, Binkley et al. (2012) have categorized the 21* century skills into four 
groupings: ways of thinking, ways of working, tools for working, and living in the world. 
Critical thinking, problem solving, and decision making; creativity and innovation; learning to 
learn and metacognition are all included in the group of ways of thinking. Communication and 
collaboration are the skills of ways of working. Tools for working contain information literacy 
and ICT literacy. Finally, citizenship (local and global), life and career, and personal and social 
responsibility are categorized under living in the world. 

Furthermore, Hixson et al. (2012) categorized the 21" skills into three domains: learning 
and innovation skills, career and life skills, and digital literacy skills. Critical thinking, 
communication, and creativity and innovation are subgroups of learning and innovation skills. 
Self-direction, collaboration, making global and local connections are included in the career 
and life skills. Finally, using technology as a tool for learning, information and communication 
technology (ICT), and media literacy are all subcategories of digital literacy skills. 

Having learners’ with 21‘ century skills is a demand for every society that wants to 
keep up with the global educational developments. Therefore, learning materials should be 
designed to support such skills. Menggo et al. (2019) conducted a wants analysis for academic- 
English speaking content in order to promote 21* century skills. First, the outcome of students’ 
desires strongly showed that material design needs to be more about 21" century skills such 
as ways of thinking that include creative and critical approaches to problem-solving and 
decision-making, ways of working that include communication and collaboration, and ways 
of using relevant technologies (ICT literacy). Furthermore, 21“ century teaching and learning 
necessitate cooperation skills by encouraging students to contribute functions, solve issues, and 
employ various thinking processes in order to attain academic success. Finally, it was advised 
that the academic-English speaking material be designed with linguistic resources to aid in 
the development of 21* century skills especially communication skills. Henceforth, it can be 
concluded that the nature of the course that is offered to the students can be a factor that might 
affect their acquisition of 21* century skills. 

Besides the learning materials, teachers also need to be able to foster 21“ century skills 
in their teaching practices in order to help their students acquire these skills. As professional 
development is the source for enhancing and influencing the knowledge and skills of teachers, 
institutions need to include what knowledge and skills teachers need in the content of their 
PDPs. Thus, in order to have 21‘ century learners, attention must be paid to how teachers can 
implement these skills in the classrooms and how they can be evaluated and trained through 
PDP. Only with successful achievement of 21st century teaching skills, 21st century learning of 
students can be enhanced and developed (Kim et al., 2019). This is also supported by Yue (2019) 
as he mentioned that higher education institutions need to address 21* century skills, such as 
problem solving and critical thinking, creativity and innovation, cross-cultural understanding, 
and ICT literacy. For this purpose, teachers need to develop all these skills and then can teach 
them to their students. As a result, in order to fulfil the 21“ century requirements, professional 
development has become an urgent need for improving successful language teaching and 
learning (Hazaea, 2019). 
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In this regard, Andrade (2016) mentioned that the learning outcomes needed for higher 
education such as communication, problem-solving, critical thinking, and innovation had a 
wide support in all institutions and should be the elements of the curricular for learner success 
in enhancing 21* century skills. Furthermore, Hasan and Parvez (2017) discussed the PD of 
teachers in higher education in India. They analyzed the aspects and areas of PDPs of teachers 
in 21“ century. It had been noticed that the universities were running updated courses with the 
help of Academic Staff in Colleges. They were largely responsible for organizing workshops 
and newly academic activities for the teachers’ PD in order to prepare teachers with the required 
skills of 21s century. Observing the classroom processes and teacher practices of enabling and 
supporting the development of 21“ century skills in the classroom is a necessary step to transfer 
these skills to the students. For example, teaching skills such as critical thinking requires 
teachers to be taught in a manner that is reflective of that process through PDPs that involve 
continuous learning and ongoing teachers’ reflection (Kim et al., 2019). 

Additionally, one of the studies that explored the issue of implementing of 21“ century 
skills in teaching practices is done by Wilcox et al. (2017). They examined the extent to which 
teachers tried to develop students’ 21 century skills in classrooms, and the extent to which 
they believed students improved those skills in their classes. They also tried to know how 
teachers effectively assessed these skills in their students. The results revealed that over third 
of the teachers employed all or part of the best instructional techniques to help students’ 21* 
century skills such as collaboration, communication, critical thinking, self-direction, and using 
technology as a tool to learn. The results also showed that all three aspects which were attempt, 
belief, and assessment could significantly predict the frequency of teaching students’ 21* 
century skills. 

Moreover, Weeks et al. (2020) examined school-based agriculture education programs to 
report the instructors’ knowledge, ability, and necessity of incorporating 21* century skills into 
the classroom. The study also attempted to provide a model for teacher education programs by 
actively encouraging the development of these skills. The 21* century skills included in this 
study were creativity, critical thinking, communication, collaboration, and technology skills. 
After assessing teachers’ perceived knowledge, and ability, the data were analyzed to establish 
the professional development requirements of school-based agriculture education instructors, 
specifically 21 century skills. The most intriguing emerging result was the simultaneous 
integration of several 21 century skills. According to the findings, professional development 
was required to better educate and equip agricultural instructors with specific and applicable 
ways for implementing critical thinking, communication, and technological literacy skills in 
their classrooms. 

On the same path, Sloan (2019) looked at how schools in an urban school system used 
professional development to help instructors incorporate 21% century survival skills into 
classroom instruction. These skills were classified as follows: critical thinking and problem 
solving, cooperation across networks and influence, agility and adaptation, effective oral and 
written communication, acquiring and analyzing information, and curiosity and imagination. 
The data gathered on how effective previous professional development was at covering these 
specific skills. According to the findings, current professional development seldom, if ever, 
included anything about survival skills. At best, there were a few instances where a skill 
was acknowledged but not explicitly shown; rather, it was more of an implicit idea. While 
community business partners and other stakeholders agreed that the skills needed to be taught, 
there was little indication that this was happening in these training sessions. Therefore, these 
survival skills need to be incorporated in teachers’ PDPs clearly and with direct instructions 
how to implement them in their teaching practices to aid students acquire and improve these 
skills. 
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In addition, Urbani etal. (2017) studied how and to what extent teacher education programs 
for Pre-service teachers developed and modeled the 21* century skills. The evaluation included 
three teacher education programs which were multiple subjects, single subject, and education 
specialist. The results indicated that assignments given to the teachers during these training 
programs greatly impacted their competence in creativity, critical thinking, communication, 
collaboration, and information, media, and technology skills and in their ability to incorporate 
these skills into their teaching practices. 

Although many educational institutions attempt to develop teachers’ activation of 
21" century skills their students’ acquisition of these skills, few studies have explored the 
improvement of teachers’ instructional practices to develop these skills after participating in 
PDPs. Furthermore, to the best of the researcher’s knowledge, there is still a lack of studies 
conducted in the Kingdom of Saudi Arabia to examine the correlation between teachers’ 
activation of 21“ century skills and their students’ acquisition of these skills. Thus, more studies 
are needed to examine how teaching practices are changed as a result of PDPs in activating 
21“ century skills and consequently developing students’ acquisition of these skills in order to 
ensure that they are capable to work after graduation. 


Research Methodology 
General Background 


This study adopted a mixed method approach with explanatory sequential design. 
This design included two phases: quantitative data collection and in-depth qualitative data 
collection. In the quantitative phase, two questionnaires were collected from EFL teachers and 
their students at Taif University English Language Center (TUELC). In the qualitative phase 
semi-structured interviews with EFL teachers were conducted as a next step to help explain the 
quantitative results. 

The current study was limited to in-service teachers who teach English as a foreign 
language (EFL teachers) at Taif University English Language Center (TUELC) in Saudi Arabia 
and their students during the second semester of the academic year 2020-2021. TUELC offers 
a PDP every semester to its EFL teachers that is why the researcher chose them to be the 
participants of this study. 

Moreover, the current study focuses in the 21“ century skills that were developed by 
Hixson et al. (2012). It includes eight dimensions which are critical thinking, collaboration, 
communication, creativity and innovation, self-direction, global connections, local connections, 
and using technology as a tool for learning. 


Sample 


There were two target populations based on the research questions. The first population 
was EFL teachers at TUELC. They are 109 from various nations and holding a variety of 
degrees. Those teachers are teaching English Language courses as general subjects to the first- 
grade students at Taif University (TU). That is why those students were also selected to be the 
second population. They are 5017 students from four Colleges at TU. Under each college there 
are 12 sub-colleges. 

In the quantitative phase, for the teachers’ population, 44 EFL teachers were selected 
randomly according to the stratified random selection of the students’ classes as shown in 
Table |. Therefore, for the students’ population, stratified random sampling was applied. First, 
the students were divided into four groups according to their main colleges. Then, they were 
divided into 12 subgroups according to their 12 sub-colleges. After that, 44 classes were selected 
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randomly according to the formula of proportionate stratified sampling technique which was 
(Sample Size*Stratum Size/ Population Size) (Nori, 2013). Finally, a random sample was 
selected within these 44 classes. 


Table 1 
The Stratified Random Selection of Classes 


Sub-college Formula Sample Number of 
g (Sample Size*Stratum Size/ Population Size) P Classes 
College of Medicine (880*90 /5017) = 15.79 20 1 
College of Pharmacy (880*91 /5017) = 16.38 20 1 
College of Applied Medical 
Sciences (880*209 /5017) = 36.66 40 2 
College of Education (880*355 /5017) = 62.26 60 3 
College of Art (880*378 /5017) = 66.30 60 3 
College of Applied Studies (880*800 /5017) = 140.32 140 7 
allege OF Shelia (880*466 /5017) = 81.74 80 4 
Regulations 
College ar ausiness (880*680 /5017) = 119.27 120 6 
Administration 
College of Sciences (880*920 /5017) = 161.37 160 8 
College of Engineering (880*89 /5017) = 15.61 20 1 
College of IT (880*462/5017) = 81.04 80 4 
College of Home Economics (880*477 /5017) = 83.67 80 4 
Total 880 44 


The sample size of the students’ population was determined based on the assumption 
of sampling table in which Cohen et al. (2007) stated that normal sampling strategy uses a 95 
percent confidence level and a 3 percent confidence interval. So, as if the population is (N) 
5000, the ideal sample size is (S) 879. The sample size of each stratum was calculated by using 
the formula of calculating a proportionate stratification as shown in Table 1. So, 20 students 
were selected randomly from each class by using simple random sampling. 

In the qualitative phase, a purposeful sampling with the criterion sampling technique is 
used in this phase to choose some cases that meet a predefined criterion for this in-depth stage 
(Palinkas et al., 2015). After examining the quantitative data, some criteria for choosing the 
participants were established. These criteria were: involving in PDPs for 2 years and more, 
having more than 5 years of teaching experience, and specializing in TESL/TESOL, English 
language, or Linguistics Studies. The majority of books and publications that Guest et al. (2006) 
studied said that the purposive sample size should be determined inductively, and that sampling 
should continue until saturation. As the saturation point was achieved at teacher number eight, 
the interview sample size was set at eight teachers. 
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Research Instruments 
The study’s data were collected using the following instruments: 


e EFL Teachers’ Questionnaire 
e Students’ Questionnaire 
e EFL Teachers’ Semi-structured Interview 


EFL Teachers’ Questionnaire 


Teachers’ questionnaire contained two sections. The first one included the respondent’s 
demographic information which was used in analyzing the data later. The second one was 
utilized to identify the extent to which participation in PDPs helps EFL teachers activate 21* 
century skills. This section was adapted from Hixson et al. (2012) and permission was sent 
by email. This questionnaire included a total of 46 items which represented the 21‘ century 
skills under eight dimensions. These items were assessed with a 5-point Likert scale with 
1=Never, 2=Rarely, 3=Sometimes, 4=Often, and 5=All the Time. The EFL teachers were 
invited to participate via email and the electronic questionnaire that was created by Google 
form was distributed online. The questionnaires were anonymous to give teachers the freedom 
to express their viewpoints; however, the group numbers were used later to facilitate the follow 
up interviews. 

In order to determine the content validity, a validation committee consisting of five 
experts from Faculty of Education and Language Center analyzed the content of the items and 
their appropriateness to the research objectives. Each member in the committee was asked to 
examine the relevance of the items to the question that is supposed to be answered, the clarity of 
the items regarding the vocabulary level, the language, and the structure, and the adequateness 
of the content which means if the number of items is representative enough of each domain in 
the questionnaire. For measuring the questionnaires’ reliability, Cronbach’s coefficient alpha 
was calculated by using SPSS program, version 23, and it yielded .955. 


Students’ Questionnaire 


The second questionnaire was applied among students, and it was the same as the 
teachers’ questionnaire, but with slight changes in the wording reflecting students’ different 
perspectives. The first change was for the heading statement. It was changed to “After studying 
English Language Course in my first semester at university, I am able to ...”. The second one was 
for the 5-point of the Likert scale. They were changed to 1= Strongly Disagree, 2= Disagree, 
3= Neutral, 4= Agree, 5= Strongly Agree. Finally, the 46 items structure was also changed to 
address the students, e.g., the item “Draw their own conclusions based on analysis of numbers, 
facts, or relevant information.” was changed to “Draw my own conclusions based on analysis 
of numbers, facts, or relevant information.” 

In order to determine the content validity, the same validation committee of teachers’ 
questionnaire was given the students’ questionnaire. Also, Cronbach’s alpha coefficient was 
used to measure the questionnaires’ reliability, and it yielded .967. 


EFL Teachers’ Semi-structured Interview 
An interview guide was created, which included a list of subjects to be covered as well 


as some open-ended questions. It was prepared following the analysis of quantitative data as 
needed by the study’s explanatory sequential design. Because of the COVID-19 epidemic, EFL 


https://doi.org/10.33225/pec/22.80.652 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) 


Mona Abdullah ALZAHRANI, Faizah NOR. Professional development and EFL teachers’ practices in activating learners’ acquisition 
of 21st century skills 


instructors were interviewed online using Zoom sessions. With the participants' agreement, all 
interviews were audio-recorded, and 30 to 45 minutes were allotted for each interview session. 
To validate the qualitative results of this study, two methods were used which were 
member checking by which the interview transcripts with the emerging themes were returned 
to four of the study participants and they provided their feedback, and a rich, thick description 
which provides more in-depth details. As for the qualitative reliability, cross-checking for the 
codes was used via asking another coder who is experienced with qualitative studies and coding 
process to cross check an interview coding and emerging themes and give his feedback. 


Research Procedures 


As discussed earlier, the current investigation used an explanatory sequential design, 
which is one of the most popular mixed methods designs in educational research (Creswell, 
2014). It includes two phases: quantitative and qualitative which was implemented sequentially 
as shown in Figure 1. 


Figure 1 
EFL Teachers’ Activation of 21% Century Skills 


Procedures 


Quantitative Data Distributing EFL Teachers’ Questionnaire 
Collection Distributing Students’ Questionnaire 


Quantitative Data 
ee Analyzing the quantitative data by using 
descriptive and inferential Analysis. 


Case Selection, Interview Selecting 8 participants from EFL teachers 
Protocol Development for interview by using Criterion Participants 
Selection. 
Developing Interview Protocol through 
choosing data that needs further explanation 
after analyzing the quantitative data. 


Conducting a semi-structured interview with 
eight EFL teachers. 


Qualitative Data 
Analysis thematic analysis via MAXQDA 2020 


Analyzing the qualitative data by using 


Quantitative & 
Qualitative Results 
Integration 


e Explaining the results by integrating 


quantitative and qualitative results. 
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Data Analysis 


Data analysis occurs in three stages in the explanatory sequential design: quantitative 
data analysis, follow-up qualitative data analysis, and how the qualitative data serves to interpret 
the quantitative results (Creswell & Clark, 2018). 

Statistical Package of Social Sciences (SPSS) and Partial Least Squares - Structural 
Equation Modelling (PLS-SEM) were used for the quantitative data analysis. SPSS was used to 
provide descriptive statistics on EFL teachers’ activation of 21st century skills, and the students’ 
acquisition of these skills. Descriptive statistics helps in organizing quantitative data and allows 
researchers to summarize characteristics and distribution of values in a single or more datasets 
(Lee, 2020). Mean and standard deviation for each statement in both questionnaires were 
calculated. 

PLS- SEM was used to analyze the data of research question 3 which aimed to identify the 
correlation between EFL teachers’ activation of 21* century skills and their students’ acquisition 
of these skills. Through SEM, researchers can explain the entire correlations between variables 
by defining a model (Kumar & Upadhaya, 2017). The SEM is more appropriate for research 
question number three in this study because it is well-known by three essential characteristics, 
namely, the assessment of many dependence relationships, illustrating unobserved concepts in 
these dependence relationships and correct the errors of measurements, and finally defining the 
whole model that explains the entire relationships between variables (Hair et al., 2014; Kumar 
& Upadhaya, 2017). 

Thematic analysis, on the contrary, was employed for qualitative data analysis since 
it delivers rich, comprehensive, and complicated data. Braun & Clarke (2006) defined it as a 
process for detecting, analyzing, and reporting themes inside the data. Thematic analysis is 
thought to be the most suited analyzing tool for any study that employs interpretations. It enables 
the researchers to link the topic frequency to the entire material (Alhojailan, 2012). Qualitative 
research thematic analysis may be broken down into five steps: joining, deconstructing, 
reconstructing, analyzing, and concluding (Castleberry & Nolen, 2018). 


Research Results 
Research Question 1: EFL teachers’ activation of 21% century skills 


To answer this question, the data were collected via questionnaire and interview. First, 
teachers were given a questionnaire. Then, eight selected teachers were interviewed to help 
explaining the quantitative results. As the questionnaire used a 5-point Likert scale, the data 
were interpreted by three levels based on the mean range as shown in Table 2. 


Table 2 
Mean Range Levels 


Interval 


Scale Length Lower Limit Upper Limit Range Level 
Never 1 
1.33 1.00 2.33 1.00 - 2.33 Low 
Rarely 2 
Sometimes 3 1.33 2.34 3.67 2.34 - 3.67 Moderate 
Often 4 
- 1.33 3.68 5.00 3.68 - 5.00 High 
All the Time 5 
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After collecting all data, the responses were processed through descriptive statistical 661 
analysis using SPSS to determine the mean and standard deviation for each item as shown in 


Table 3. 


Table 3 
Mean and Standard Deviation for the Items of EFL Teachers’ Activation of 21st Century 
Skills 


Dimension N M SD 
1 3.37 1.036 
2 3.53 1.077 
Critical thinking skills 2 = sa 
4 3.57 0.996 
5 3.47 1.055 
6 3.68 1.017 
7 4.40 0.739 
8 3.74 1.094 
Collaboration skills 9 3.49 1.033 
10 4.18 0.971 
11 4.06 1.114 
12 3.46 1.129 
13 3.82 1.116 
Communication skills 14 3.91 0.936 
15 3.97 1.005 
16 4.15 1.073 
17 4.00 0.976 
18 3.91 1.019 
Creativity and innovation skills 19 3.66 1.055 
20 3.51 1.066 
21 3.52 1.130 
22 3.46 1.076 
23 3.17 1.241 
24 3.51 1.130 
Self-direction skills 25 3.15 1.317 
26 3.67 1.148 
27 3.52 1.199 
28 3.52 1.209 
29 3.83 1.037 
30 3.91 0.936 
Local connections 31 3.14 1.287 
32 2.84 1.247 
33 3.13 1.283 
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34 4.00 0.915 
35 3.75 0.918 
Global connections 36 3.34 1.076 
37 3.83 1.002 
38 3.45 1.108 
39 4.09 1.063 
40 4.06 0.981 
41 3.59 1.177 
Using technology as a tool for learning - a ne 
43 4.02 1.034 
44 4.03 0.994 
45 3.10 1.414 
46 3.67 1.158 


Table 3 shows that the mean score range for all items in the questionnaire is 2.84 till 
4.40. It could be seen that Item 7, which states “Working in pairs or small groups to complete 
a task together.”, received the highest mean score of 4.40 (SD= 0.739). This skill belongs to 
the Collaboration Skills dimension. The high mean score for this item signifies that teachers 
activate this skill more than the other skills. It means that teachers always encourage their 
students to work in pairs or in groups while completing a task. Teachers’ choice of this item 
as the highest skill activated in the classrooms could be explained based on their expressions 
during the interview as follows. 


“I used to divide my students into groups and ask them to work in groups, and my job is to monitor 
these groups, just to make sure that they work together and communicate in English. I praise them 
when they when they come up with an innovative idea especially low-level student.” (Collabora- 
tion skills > Working in groups> T1). 


“Yes, of course, group works, I'm trying my best to make them work in groups because we teach 
large number of students. This let them share their ideas and learn from each other.” (Collabora- 
tion skills> Working in groups> T3). 


On the contrary, the item that has the lowest mean is Item 32 with a mean score of 2.84. 
It states that “Analyzing how different stakeholder groups or community members view an 
issue.” and belongs to the dimension of Local Connections. This result means that teachers 
seldom teach their students how to analyze different views of local community members. This 
may be due to the topics included in English courses which are derived mostly from global 
communities as Teacher 7 mentioned: 


“The course let the students make Global Connections. The topic like for example Amazon is 
not something that we always talk about like Ganaderiah or other local stuff; or camel race or 
something like that. This is a different time in our lives. This is truly a global time in our lives and 
our learners need to get used to that and trained for that because they learn a global language.” 
(Activation of 21" century skills > Local and global connections> T7). 


Therefore, the kind of topics included in the textbook could be a reason why the skill 
of analyzing different views of local community is not always activated by teachers in EFL 
classrooms. Sometimes, teachers depend only on the topics of English textbook and do not try 
to connect the global issues with the local ones. 
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Upon generating the means for individual items, the items were then computed according 
to the eight dimensions of 21* century skills as shown in Figure 2. 


Figure 2 
EFL Teachers’ Activation of 21% Century Skills 
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Figure 2 displays the mean scores for all eight dimensions which range from 3.37 to 
3.97. The dimension of Collaboration Skills has the highest mean score with 3.97. This result 
means that EFL teachers activated collaboration skills in the classroom all the time. 

On the other hand, Local Connections got the lowest mean score with 3.37. According 
to the mean range in Table 2, this result means that EFL teachers sometimes activate this skill. 

The reading of the results for all dimensions of 21“ century skills as presented in Figure 
2 shows that PDPs help teachers to activate all 21‘ century skills in a moderate and high level. 
This result is supported by teachers’ interview as follows. 


“Yes, and most of them actually, like, for example, the use of technology inside classroom. In some 
training sessions, we were presented to a lot of apps and websites that we can use inside the class. 
Also, for the critical thinking skills.” (Activation of 21" century skills> T2). 


“PDPs presented strategies that teacher can use to activate these skills inside the classroom. 
Many strategies, and many techniques that I have learned and applied in my classroom. They 
were talking about critical thinking, collaboration, and self-direction. So, these training sessions 
talked about these skills, and we got benefit of it.” (Activation of 21" century skills> T5). 


The excerpts above indicate that PDPs help teachers be aware of 21‘ century skills and 
how to implement them inside their classrooms. T2 mentioned that some training sessions 
provide her with apps and websites that enabled her to activate the skills of using technology. 
In addition, TS added that she has learned many strategies and techniques from PDPs which 
helped her in activating skills such as critical thinking, collaboration, solving problems, and 
self-direction skills. 


Research Question 2: Students’ acquisition of 21" century skills 


The students’ acquisition of 21" century skills was measured through students’ 
questionnaire and their teachers’ semi-structured interviews. The questionnaire uses a 5-point 
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664| Likert scale with options, Strongly Disagree= 1, Disagree= 2, Neutral= 3, Agree= 4, Strongly 
Agree=5. Thus, the higher the value, the larger the extent of 21st century skills acquisition by 
the students. The data of this question was interpreted by three levels based on the mean range 
as shown in Table 4. 


Table 4 
Mean Range Levels 


Interval 


Scale Length Lower Limit Upper Limit Range Level 
Strongly Disagree 1 
; 1.33 1.00 2.33 1.00 - 2.33 Low 
Disagree 2 
Neutral 3 1.33 2.34 3.67 2.34 - 3.67 Moderate 
Agree 4 , 
1.33 3.68 5.00 3.68 - 5.00 High 
Strongly Agree 5 


As shown in Table 4, there are three levels: low, moderate, and high by which the means 
will be interpreted. After that, mean and standard deviation for each item in the questionnaire 
were produced. The results are presented in Table 5. 


Table 5 
Mean and Standard Deviation of Students’ Questionnaire 


Dimension N M SD Total M and SD 
1 3.77 0.931 
2 3.80 0.864 

7 _ 3 4.04 0.895 M= 3.83 

Critical thinking skills 4 3.86 0.896 SD = 0.663 
5 3.75 0.930 
6 3.77 0.966 
7 4.07 0.988 
8 3.97 0.995 

Collaboration skills 9 3.87 0.929 Pe 2 are 
10 3.98 0.965 
11 3.74 1.063 
12 3.84 0.958 
13 3.98 0.900 

Communication skills 14 4.05 0.891 ie ea 
15 3.59 1.141 
16 3.63 1.137 
17 3.86 0.937 
18 3.95 0.842 

Creativity and innovation skills 19 3.92 0.870 : ae 

SD =0.725 

20 3.92 0.854 
21 3.97 0.884 
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22 4.13 0.884 
23 4.02 0.851 
24 4.04 0.887 
Self-direction skills 25 3.93 0.889 ae 
26 4.00 0.900 
27 3.91 0.896 
28 4.11 0.879 
29 3.65 0.990 
30 3.87 0.868 
Local connections 31 3.63 0.967 ] ee 
32 3.58 0.909 
33 3.81 0.854 
34 3.75 0.988 
35 3.71 0.934 
Global connections 36 3.50 1.023 ae 
37 3.85 0.942 
38 3.57 1.020 
39 4.22 0.894 
40 4.12 0.837 
41 3.95 0.921 
Using technology as a tool for learning 2 ule wally ee 
43. 4.19 0.850 SD = 0.716 
44 4.09 0.884 
45 3.98 0.914 
46 4.10 0.906 


It can be seen from Table 5 that all items have mean scores ranged between 3.47 and 
4.22 which represent moderate and high levels of agreement. Item 39 received the highest 
mean with a mean score of 4.22. Item 39 states that “Using technology or the Internet for self- 
instruction (e.g., videos, tutorials, self-instructional websites, etc.).” This finding reveals that 
the students acquired the skill of using technology for self-learning in their English classes. This 
skill belongs to the domain of Using Technology as a Tool for Learning which also received the 
highest mean score (4.08) among the eight dimensions of 21* century skills. This result is also 
confirmed by teachers’ responses during the interview as follows. 


“They did an amazing work. Even their videos when I asked them to do a speaking task and send 
me a record or something of their speaking, they would send me videos that amazed me.” (Acqui- 
sition of 21" century skills)> Using Technology> T1). 

“T'm so happy that they are very skilled with technology and using it. You know, if I ask them to do 


anything, they were doing it in a second and I was amazed.” (Acquisition of 21" century skills) > 
Using Technology>T3). 


Teachers mentioned some reasons behind this result in the interview as follows. 
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“T think online classes during COVID-19 Pandemic enhanced some of the 21" century skills such 
as using technology. The online classes were more effective in helping the students learn how to 
use technology because they are forced to use it.” (Using Technology > Online classes> T2). 


“I feel like some skills have been developed through online classes, especially technology. Stu- 
dents started to send me everything through email, they started to share with me on Drive, and 
create their own videos. They also started to use different websites for large size files.” (Using 
Technology > Online classes> T4). 


The above excerpts show that online classes helped students to acquire skills using 
technology. Teaching and learning processes have been shifted from face-to-face into online 
classes due to COVID-19. 

In contrast, the item that received the lowest mean is Item 36 with a mean score of 3.50. 
Item 36 states that “Discussing issues related to global interdependency (for example, global 
environment trends, global market economy.)” This result reveals that the students agree with 
this statement in a moderate level. It means that they rarely discuss the global interdependency 
issues. This skill is related to the domain of Global Connections which also received the lowest 
mean score (3.67) among the eight dimensions of 21“ century skills. 

In the interview when teachers were asked about the skills that their students acquired, 
they mentioned all the skills except global connections. The following are some examples of 
their responses. 


“Using technology. This is number one, because they all had to use their devices regardless of 
what device they are using. May be local connections because they had to be connected to the 
teacher. Also, communication and collaboration with each other and with their teachers. Self-di- 
rection skills, maybe but not in very high rates” (Students’ Acquisition of 21" skills> T6). 


“Some courses help in developing critical thinking, they actually challenge students’ critical 
thinking. So, some students have that critical thinking and innovation because of the courses they 
are learning.” (Students ’ Acquisition of 21" skills> T7). 


It can be noted from the excerpts above that teachers 6 and 7 talk about the students’ 
acquisition of most of 21% century skills such as using technology, local connections, 
communication and collaboration, self-direction skills, critical thinking, and innovation. None 
of the teachers have pointed to global connections as an acquired skill by the students. 


Research Question 3: The correlation between EFL teachers’ activation of 21" century skills 
and their students’ acquisition of these skills 


To explore the correlation between teachers’ activation of 21“ century skills and their 
students’ acquisition of these skills, measurement and structural model assessments were 
applied via PLS-SEM. 


Measurement Model Assessment 
The measurement model assessment was evaluated by addressing Construct Reliability 
using Composite Reliability (CR) and Cronbach’s Alpha, and Convergent Validity using Outer 


Loadings and Average Variance Extracted (AVE) (Hair et al., 2017). Figure 3 presents the PLS 
Algorithm results for the assessment of the measurement model. 
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PLS Algorithm Results for the Measurement Model Assessment 
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Construct Reliability: Composite Reliability (CR) and Cronbach Alpha. Cronbach’s 
Coefficient alpha shows the average correlation among all the items of the scale. According 
to Pallant (2016), the acceptable value of Cronbach’s alpha is from 0.70 to 0.95. Regarding 
the Composite Reliability (CR), Nunally and Bernstein (1994, cited in Hair, Hult, Ringle, & 
Sarstedt, 2017) pointed out that the values ranging between 0.70 and 0.90 were acceptable 
values of CR. Table 6 presents the values of Cronbach’s alpha and composite reliability. 
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Table 6 
Results of Cronbach’s Alpha and Composite Reliability (CR) 


Constructs Cronbach's Alpha Composite Reliability 
Students’ Critical Thinking 820 866 
Students’ Collaboration 858 899 
Students’ Communication 832 881 
Students’ Creativity and Innovation 885 916 
Students’ Self-direction 919 935 
Students’ Local Connections 831 881 
Students’ Global Connections 865 .902 
Students’ Using Technology 922 937 
Teachers’ Critical Thinking 842 888 
Teachers’ Collaboration 120 801 
Teachers’ Communication 113 815 
Teachers’ Creativity and Innovation 848 893 
Teachers’ Self-direction 805 857 
Teachers’ Local Connections 823 877 
Teachers’ Global Connections 821 873 
Teachers’ Using Technology 819 863 


Table 6 presents the values of Cronbach’s alpha and composite reliability for all 
constructs. The results show that all the values of all constructs are within the acceptable value 
(0.70 — 0.95) according to Pallant (2016). This means that the items of these constructs are 
internally consistent. 

Convergent Validity: Outer Loadings and Average Variance Extracted. In order to test 
the convergent validity, the outer loading of the indicators and the average variance extracted 
(AVE) from the constructs should be considered. According to Hair et al. (2014) the value of 
0.50 and above are considered significant. Therefore, the indicators with low outer loadings 
must be removed from the scale. AVE is the other common measure to test the convergent 
validity of the constructs level. Hair et al. (2017) cited that the AVE value of 0.50 and above 
is acceptable. Besides, Fornell and Larcker (1981) stated that if AVE is less than 0.50 but 
the composite reliability is higher than 0.6, the convergent validity is still acceptable. Table 7 
presents the outer loadings and the AVE of students’ and teachers’ questionnaires. 
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Table 7 


Outer Loadings and AVE of Students’ and Teachers’ Questionnaires 


. . Students Teachers 
Dimensions Items 
Loading (>0.5) CR AVE Loading(>0.5) CR AVE 
cl 0.629 0.750 
cT2 0.735 0.710 
ritical Thinking Skill CT3 0.687 0.866 0.366 Deleted 
(cn. _— CT4 0.790 0828 0.849 most 
CTS 0.769 0.830 
CT6 0.740 0.768 
COL1 0.812 0.585 
COL2 0.853 0.770 
oa oul COL3 0.838 0899 0.641 an ; meOt “age 
COL4 0.824 0.749 
COL5 0.663 0.723 
COM1 0.777 0.635 
a com2 0.786 0.539 
eas alls COM3 0.796 0.881 0.596 0.681 DBE g580 
coMm4 0.77 0.725 
COMS5 0.723 0.824 
CR&IN1 0.794 0.676 
” CR&IN2 0.849 0.871 
oad CR&IN3 0.848 0.916 685 0.750 0.893 0.626 
(CR&IN‘) CR&IN4 0.823 0.830 
CR&IN5 0.822 0.816 
SDS1 0.825 0.725 
SDS2 0.796 0.601 
a SDS3 0.829 0.625 
ao SDS4 0.842 0.935 0.674 0.792 0.857 0.466 
SDS5 0.839 0.805 
SDS6 0.801 0.538 
SDS7 0.815 0.646 
Lc1 0.749 0.563 
Lc2 0.794 0.749 
Local Connections (LC) LC3 0.745 0881 0.596 0.838 Oem “yen3 
LC4 0.773 0.840 
LC5 0.798 0.825 
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GC1 0.796 0.645 
GC2 0.816 0.738 
co Connections 6C3 0.809 0.902 0.649 0.823 0.873 0.580 
GC4 0.824 0.809 
GC5 0.783 0.780 
UT1 0.813 0.717 
UT2 0.835 0.702 
UT3 0.752 0.635 
Using Technology asa _UT* 0808 937 0.649 sil 0.863 0.443 
Tool for Learning (UT) UT5 0.844 0.658 
UT6 0.819 0.689 
UT7 0.771 0.545 
UT8 0.800 0.717 


Table 7 presents the outer loadings for the items that belong to the variable of students’ 
acquisition of 21st century skills. The results indicated that there was no deleted item because 
the outer loadings for all items were more than 0.5. Also, the results from the table above 
indicated that all the dimensions of the 21st century skills had high values of AVE. All values 
were 0.50 and higher, which means that the dimensions clarify more than half of the variance 
of their indicators. 

Furthermore, Table 7 indicates that two items from teachers’ questionnaire have been 
deleted because their outer loadings were less than 0.5. These items were CT3 and COL3. 
Additionally, the results from Table 8 displayed that all the constructs of teachers’ activation 
of 21st century skills had acceptable values of AVE according to Fornell and Larcker (1981). 
All values were 0.50 and higher except for the Collaboration Skills, Self-Direction Skills, and 
Using Technology as a Tool for Learning in which AVE was less than 0.50 but their CR was 
higher than 0.6. 


Structural Model Assessment 
Assessment of Collinearity (VIF Values). This assessment is conducted to check if there 
are critical levels of collinearity between every set of the predictor variables. Each construct 


should have a VIF value below (5.) and any threshold value higher than (5.) means there is a 
collinearity problem (Hair et al., 2017). The VIF values were checked and presented in Table 8. 
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Table 8 

Values of VIF 
Dimensions VIF 
Student Critical Thinking 1000 
Student Collaboration 1000 
Student Communication Skills 1000 
Student Creativity and Innovation Skills 1000 
Student Self-Direction Skills 1000 
Student Local Connections 1000 
Student Global Connections 1000 
Student Using Technology 1000 
Teacher Critical Thinking 1000 
Teacher Collaboration 1000 
Teacher Communication Skills 1000 
Teacher Creativity and Innovation Skills 1000 
Teacher Self-Direction Skills 1000 
Teacher Local Connections 1000 
Teacher Global Connections 1000 
Teacher Using Technology 1000 


The results from Table 9 find that all values of VIF are below 5. This means that there is 
no collinearity problem because collinearity does not reach critical levels in any construct of the 
model. Therefore, the values of VIF in the current study are acceptable. 

Path Coefficients of Structural Model. The values of path coefficients that are close to 
+1 indicate a strong positive statistically significant correlation between the constructs. The 
values of path coefficients that are close to -1 refer to the strong negative statistically significant 
correlations between the constructs. The values that are very low and close to 0 are commonly 
considered not significantly different from 0 (non-significant) (Sarstedt et al., 2017). The results 
of the path coefficient (f) for the direct correlation of the structural model are presented in Table 
9: 


Table 9 
Structural Model Results: Direct Correlation 


Path Coefficient 
(8) 
Teachers’ Activation of 21st Century Skills -> Students’ Acquisition of 21st Century Skills 0.024 


Direct Correlation between Variables 


The results from the table above indicate that Teachers’ Activation of 21‘ Century 
Skills has no significant direct correlation with Students’ Acquisition of 21‘ Century Skills as 
the standard coefficient (B) (0.024) is below | and close to 0. Therefore, there was no direct 
correlation between the tested variables. 

As the previous result looked at the correlation as a whole, the correlation in every 
dimension of the 21st century skills was also examined using Pearson Correlation Coefficient 
via SPSS. The interpretation of Pearson Correlations was based on description of correlation 
strength as follows: < .20 very weak, .20 - .39 weak, .40 - .59 moderate, .60 - .79 strong, >.80 
very strong (Lehman, 2005). Table 10 shows the results Pearson Correlation Coefficients. 


ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) https://doi.org/10.33225/pec/22.80.652 


PROBLEMS 
OF EDUCATION 


IN THE 21*CENTURY 
Vol. 80, No. 5, 2022 


671 


PROBLEMS 
OF EDUCATION 


IN THE 21*CENTURY 
Vol. 80, No. 5, 2022 


672 


Mona Abdullah ALZAHRANI, Faizah NOR. Professional development and EFL teachers’ practices in activating learners’ acquisition 
of 21st century skills 


Table 10 
Pearson Correlation Coefficients Results 


Pearson Correlation 


21s Century Skills 


r p- 
Critical Thinking Skills 237" .0001 
Collaboration Skills .135** .0001 
Communication Skills .086* 011 
Creativity &lnnovation Skills .074* .028 
Self-Direction Skills -.055 104 
Local Connections -.059 .080 
Global Connections .086* 011 
Using Technology as a Tool for Learning 023 495 


** Correlation is significant at the .01 level (2-tailed). 


* Correlation is significant at the .05 level (2-tailed). 


Table 10 presents the results of Pearson Correlation Coefficients for the teachers’ 
activation of 21st century skills and their students’ acquisition of these skills. It could be seen 
that there was a significant correlation for the category of Critical Thinking Skills (7 = .237, 
p < .01), Collaboration Skills (r = .135, p < .01), Communication Skills (7 = .086, p < .05), 
Creativity & Innovation Skills (r = .074, p < .05), and Global Connections (7 = .086, p < .05). 
As the correlations values were less than 0.20, the strength of the correlations was very weak 
in all these skills except for critical thinking in which the correlation value was more than 0.20 
which indicated a weak correlation. These findings revealed that the students’ acquisition of 
these skills was likely linked with their teachers’ activation of these skills. The more teachers 
implemented these skills inside the classrooms, the more their students can acquire them. 

On the other hand, there was no significant correlation for the category of Self-Direction 
Skills, Local Connections, and Using Technology as a Tool for Learning. This means that the 
students’ acquisition of these skills had no correlation with their teachers’ activation of these 
skills. 


Discussion 


The results of the first question revealed that teachers have activated all 21“ century 
skills dimensions with different levels of activation. The two dimensions that got the highest 
level of activation were the collaboration and communication skills. This means that most of 
the training sessions have incorporated these two skills more than other skills. It also signifies 
that teachers activate this skill in English classrooms almost all the time. This means that 
teachers always encourage their students to work in pairs or in groups while completing a task. 
They use this skill to facilitate learning English through group working and exchanging ideas 
between students. This kind of interacting helps students in higher education to learn the skills 
of collaboration which indeed enhances their learning and social engagement (Menggo et al., 
2019). This result is consistent with Wilcox et al. (2017) in which collaboration skills were the 
most frequently used inside the classrooms to enhance students’ 21* century skills. 

On the other hand, the two least activated dimensions were the local connections and the 
self-direction skills. This reveals that the training sessions of PDPs paid little attention to those 
two skills. Therefore, the designers of these programs should concentrate on all the dimensions 
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of 21* century skills with the same amount of focus in order to avoid concentrating on some 
dimensions and neglecting the others. It also indicates that teachers seldom teach their students 
how to analyze different views of local community members. Therefore, they need to pay more 
attention to it as this is one of the most important life and career skills for adults as categorized 
by Hixson et al. (2012). 

The reading of the results for all dimensions of 21* century skills shows that PDPs 
help teachers to activate all 21* century skills in a moderate and high level. This result is also 
supported by teachers’ excerpts from the interview. They indicated that PDPs helped them be 
aware of 21“ century skills and how to implement them inside their classrooms. This is in line 
with Alamri’s (2020) study that emphasized the implementation of professional development 
courses to enhance the activation of 21“ century skills. Besides, Garcia (2021) mentioned ten 
areas that EFL teachers should focus on to make their instruction in line with 21“ century 
approaches. One of these areas was teachers’ professional development. This means that PDPs 
can promote teachers’ implementation of 21“ century skills. 

The results of the second question demonstrated that students acquired all 21* century 
skills with different levels. The dimension that got the highest level of acquisition was using 
technology as a tool for learning. This means that teachers encouraged their students to use 
technology inside and outside the class when they were asked to complete certain home tasks. 

One reason behind acquiring the skill of using technology more than other skills is 
that leaners have been shifted to be digital generation as mentioned by Yang et al. (2021). 
Therefore, it was easy for them to implement the technical skills especially in higher education. 
It is also mentioned in the study of Manuel et al. (2021) that Information and Communication 
Technology (ICT) is a crucial life’s tool in the 21“ century. That is why current learners pay 
more attention to it. 

The other reason is the emerging trends of online learning as a result of COVID-19 
pandemic. Yang et al. (2021) mentioned that the digital learning abilities are becoming a key for 
learners’ success in the period of pandemic as the students learn independently. Teaching and 
learning processes have been shifted from face-to-face into online classes due to COVID-19. 
This result is supported by Vargo et al. (2021) as they concluded that most of the instructors 
and students chose to use online classes and platforms to continue their education during the 
pandemic. Furthermore, the study of Batez (2021) revealed that students evaluated their level 
of ICT skills as being higher for online education during COVID-19 period. 

On the other hand, the least acquired dimension by the students was global connections. 
This result means that they rarely discussed global issues inside English classrooms. This result 
is in line with Ghaith’s (2010) study where he stated that the awareness about global and cross- 
cultural issues was ranked as the lowest achieved skill in higher education, and consequently 
needed more focus in different courses instructions. 

The third research question involved finding out if there was a correlation between 
teachers’ activation of 21“ century and their students’ acquisition of these skills. The results 
of the standard coefficient (B) in the SEM revealed that teachers’ activation of 21“ century 
skills had no significant direct correlation with students’ acquisition of 21% century skills. 
Accordingly, this result was in line with Fischer et al. (2018) which demonstrated that teachers’ 
engagement in PDP was favorably connected with instructors’ classroom practice. However, 
the observed parts of instructional practices had a very minor effect on students’ performance in 
the predicted direction. This indicated that participation in PDP might influence instructors to 
improve their teaching practices, but the correlation between instructional aspects and student 
achievement was very weak. 

Lastly, Pearson Correlations Coefficients were used to look at the correlation in every 
construct (dimension) of the 21‘ century skills. The results showed that there was a significant 
correlation for the category of Critical Thinking Skills, Collaboration Skills, Communication 
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Skills, Creativity & Innovation Skills, and Global Connections. This means that the students’ 
acquisition of these skills was likely linked with their teachers’ activation of these skills. Cinar 
(2021) claimed that every component of 21* century skills become a requirement for English 
language instruction in order to promote students’ requirements and learning differences. 

On the contrary, there was no significant correlation for the category of Self-Direction 
Skills, Local Connections, and Using Technology as a Tool for Learning. This means that the 
students’ acquisition of these skills had no relationship with their teachers’ activation of these 
skills. This result supported the idea that learning approach nowadays is learner-centered in 
which students learn by themselves. Also, online learning helped them to develop their self- 
direction skills without depending too much on their teachers. A study has been conducted by 
Gomez-Parra (2021) on the development of the 21* century skills via face-to-face or online 
teaching. The results revealed that autonomy skills have been developed better through online 
learning. However, teachers need to practice these skills with their students inside the classrooms 
to help them develop these skills. Yue (2019) stated that higher education institutions need 
to focus on 21* century skills, such as problem solving and critical thinking, creativity and 
innovation, cross-cultural understanding, and ICT literacy. 


Conclusions and Implications 


This study aimed to explore the influence of PDPs on EFL teachers’ practices in 
activating students’ acquisition of 21“ century skills. It also aimed to examine the correlation 
between teachers’ activation of 21* century skills and their students’ acquisition of these skills. 
The data were collected using tow questionnaires and semi-structured interviews. The results 
revealed that teachers have activated all 21* century skills dimensions with different levels of 
activation. Moreover, the results demonstrated that students acquired all 21* century skills with 
different levels too. Furthermore, the results of the standard coefficient ($) in SEM revealed 
that teachers’ activation of 21* century skills had no significant direct correlation with students’ 
acquisition of 21* century skills. Lastly, the results of Pearson Correlations Coefficients which 
were used to look at the correlation in every dimension of the 21“ century skills, showed that 
there was a significant correlation in the category of Critical Thinking Skills, Collaboration 
Skills, Communication Skills, Creativity & Innovation Skills, and Global Connections. On the 
other hand, there was no significant correlation in the category of Self-Direction Skills, Local 
Connections, and Using Technology as a Tool for Learning. 

This study provides theoretical and practical implications for various stakeholders related 
to PDPs in higher education, namely PDPs designers, university administration, and EFL 
teachers. Since it was found that PDPs have influenced teachers’ practices, it is recommended 
to offer online PDPs during flexible times besides the onsite ones. Some teachers do not 
have enough time to go for onsite trainings. Thus, offering online trainings allows teachers 
to participate in settings that suit their work time and lifetime. Therefore, teachers will focus 
more on the presented training sessions. So, universities should pay attention to their teachers’ 
workload and their free time to plan the PDPs timings accurately. Teachers need to attend 
these programs with fresh minds in order to focus and learn in a way that enables them to 
transfer what they have learned to their classrooms to enhance their students’ achievement. 
Additionally, universities are in consistent need for additional funding to support PDPs and 
make it as an ongoing lifelong process for their teachers. 

Furthermore, as we live in a globalized, technological world, PDPs developers at higher 
education should integrate 21“ century skills that help students to cope with this developed era. 
Curricula plays an important role in facilitating the acquisition of these skills as resulted in this 
study. Therefore, having such skills in the students’ curricula will help teachers enhance them 
and then acquire by the students. The 21‘ century skills framework on which this study was 
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based was developed with input from teachers, education experts, and business leaders to define 
and explain the skills and knowledge students need to be successful in their life and work, and 
to illustrate the support systems that are necessary for 21‘ century learning outcomes. Thus, 
all the eight dimensions that were included in this study should be learned and acquired by the 
students to be prepared well for their future career. 

Since this study did not examine gender differences among teachers and students at 
higher education and whether these differences have an impact on EFL teachers’ educational 
practices in enhancing students’ acquisition of the 21“ century skills, it is worthwhile to study 
this issue further. Moreover, the current study covered all the eight dimensions of the 21* 
century skills as reported in the framework of 21“ century skills. Thus, it would be interesting 
to conduct a study that concentrates on one dimension of the 21“ century skills. This will make 
the findings more detailed and focused. 

To sum up, English language teachers need to be professionally competent with the latest 
changes in this 21“ century. Changes in the educational environment have made teacher training 
programs complex and difficult. Therefore, it is hoped that teachers’ responses in this study 
regarding the activation of 21* century skill provide the administrators with recommendations 
about ways to implement 21* century skills in their PDPs. This feedback may help the PDPs 
planners to pay more attention to the skills on which teachers need more training sessions. 

In addition, as the students at university are getting ready for the workplace, they need 
to be well equipped with knowledge and skills which will enable them to succeed in their work 
and life. One of these skills is 21* century skills. Thus, this study may provide EFL teachers 
an opportunity to better understand the importance of enhancing students’ 21* century skills to 
prepare them for the future challenged life. 
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Abstract 


Artificial Intelligence (Al) is changing the way people live in society. New technologies powered by AI 
have been applied in different sectors of the economy and the educational context is no different. AI has 
been considered a key to the development of learning strategies, especially in distance education. In this 
sense, this research aimed to identify the current state of Brazilian literature on AI applied to distance 
education. The Higher Education market in Brazil, which is the biggest in Latin America regarding the 
number of individuals able to enroll in a program, is still developing and distance education has grown 
rapidly. To reach the purpose of this paper, it was performed a Systematic Literature Review (SLR) to 
find the research conducted in graduate programs that investigate the subject of AI applied to distance 
education. The final analysis used a total of 63 studies — 26 master s theses and 37 doctoral dissertations. 
The main results show that most of the research on AI in distance education in Brazil was conducted in 
Computer Science (56%) and Engineering (27%). Only 6% of the studies reviewed are from masters’ or 
doctoral programs in Education. The result also shows that limited attention is paid to critical topics 
related to the growing introduction of AI in distance education, as such teachers’ employability and 
technological training or the ethical implications of using AI for the educational process. As a result of 
this SLR, it was possible to suggest research opportunities considering the international agenda on AI. 
Keywords: artificial intelligence, distance education, educational technology, systematic literature 
review 


Introduction 


Each period is marked by a technological development that seeks to meet the needs 
of individuals. New solutions are proposed and implemented to solve the existing problems, 
facilitating and improving some aspects of people’s lives. In the last decades, society has 
witnessed technological revolutions in several sectors, impacting production, sales, and jobs 
(Albuquerque, 2019). The so-called 4" Industrial Revolution (4TR) has been the focus of 
analysis of several researchers (Schwab, 2017; Xu et al., 2018) and it is considered a strategy 
to improve competitiveness in business production (Gazquez et al., 2021). Described in 2016 
by Klaus Schwab, the founder of the European Symposium of Management, the precursor of 
the World Economic Forum, the 4IR refers to the context in which the offline world interacts 
directly and constantly with the online world. 

Powered by innovations arising from the Internet of Things (IoT), big data, and AI, 
among other innovations, the 4IR is already showing its impacts on the job market in several 
areas (Eberhard et al., 2017). According to specialists, the current technological changes, 
unlike previous revolutions, may not generate new jobs in the same proportion as they reduce 
the existing occupations (Gleason, 2018). Researchers have drawn attention to the risk of the 
implementation of these new technologies creating a mass of qualified unemployed people 
around the world (Bessen, 2020; Peters, 2017; Peters & Jandric, 2019, Teng et al., 2019). 
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The more standardized the task is, the more likely it is to be replaced by AI technology 
(Eberhard et al., 2017). AI has different definitions, depending on the area in which it takes 
place. Some experts define it as the creation of intelligent agents, which represent systems 
that perceive their environment and perform actions aimed at maximizing the cases of success 
(Russel & Norvig, 2019). Others consider AI as the developed technology that provides 
interaction between humans and machines, either by coupling artificial intelligent systems to 
the biological organism or by the virtual connection between these agents (Kaufman, 2020). It 
is also defined by the literature as the study of how to make computers do things that humans 
are currently better at (Ertel, 2018). 

Despite the bias that many researchers have of transferring a human aspect to AJ, it is 
important to clarify that all technologies represent the product of human ideas (Pinto, 2005). 
They are created by individuals to solve the challenges that they face when in contact with 
the environment. Therefore, it is not possible to understand AI as an alive agent that has its 
own will. The claim that robots will control humanity only belongs on the movie screens. 
Nevertheless, AI might be implemented in some production processes by the desire of a specific 
group that can generate adverse consequences for the masses. However, in this context, it is 
not the technology that has the power of destruction, but rather a privileged social group that 
controls the productive forces. 

AI has been used in the educational setting for more than 30 years, but little is known 
about the effects of these technologies on teaching and learning in higher education. Previous 
surveys have shown that most of the research about AI comes from Computer Science and deals 
with its application in four areas: 1) Profiling and Prediction; ii) Assessment and Evaluation; 
iii) Adaptive Systems and Personalization; and iv) Intelligent Tutoring Systems (Zawacki- 
Richter et al., 2019). Among the positive points that came with application technology in higher 
education is the greater support and tutoring for students. On the other hand, there are still 
controversial points about the use of AI in education such as the ethical problems regarding the 
use of machines, the threat to the employability of teachers (Peters & Jandric, 2019; Wogu et 
al., 2019; Zawacki-Richter et al., 2019) and the mismanagement of educational data to power 
Al (Figaredo, 2020). 

In distance education, in particular, AI has more applicability. This is because the 
operation of distance learning in its essence requires a basic technological structure that can 
be easily combined with AI through Machine Learning (ML) techniques and more specifically 
with Deep Learning (DL) methods (Yu et al., 2017). DL can be understood as a representation 
learning method that allows the ML use of complex and multi-layer representations (LeCun 
et al., 2015). In this sense, the broad set of information available in online environments, such 
as the time spent by students on materials, participation in forums, and grades, among other 
possibilities, represents a great opportunity for the introduction of AI for various purposes. 

In addition, the application of AI in distance learning has been considered important to 
improve teachers’ performance. Since 2016, with the first assistant teacher created through AI 
to assist in the educational process in a course at the Georgia Institute of Technology (Goel & 
Polepeddi, 2016), many tutoring proposals carried out by machines have been applied around 
the world, from application in the assistance provided on online forums to the evaluation of 
activities (Kim et al., 2020). AI defenders argue that these technologies can free teachers from 
basic manual tasks, allowing them to dedicate themselves to activities that demand greater 
critical capacity and creativity (Goel & Polepeddi, 2016). 

However, recent evidence around the world suggests that the number of teachers may be 
reduced by the implementation of AI technology (Kim et al., 2020; Melim & Moraes, 2021). 
Technological innovations represent a possibility of reducing costs, which makes the introduction 
of AI for profit-based private Higher Education Institutions (HEI) a must for the maintenance 
and/or increase of economic results (Ramble, 2014). For HEIs that belong to conglomerates 
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with shares traded on the market, the pressure for the adoption of AI technologies may occur 
even more strongly, given the need for accountability to shareholders. 

In Brazil, the educational context may encourage HEI to introduce AI strategies. Brazilian 
higher education is the largest in Latin America with almost two million students completing 
high school each year and who can enroll in an undergraduate program (MEC, 2022). The 
Brazilian government’s goal for 2024 is to achieve a gross enrollment rate in higher education 
of 50% and a net rate of 33%. In 2019, these parameters were 37.4% and 21.4%, respectively. 
In this sense, the distance education modality has been used by the Brazilian Government to 
accelerate the number of graduates and AI has been considered essential for reaching a greater 
number of students and maintaining the quality of education (Torres et al., 2019). In 2020, for 
the first time, the number of freshman students on the distance education model was higher than 
the face-to-face undergraduate programs in Brazil (MEC, 2022). 

In this way, the general research question that guided this study was: What is the current 
state of research on AI that is applied to distance education in Brazil? In this sense, the objective 
of this paper was to identify the current state of Brazilian literature on AI applied to distance 
education. To do this, it was conducted an SLR, based on the following questions: 


(1) Where does Brazilian research on AI in distance education come from? 

(2) What are the main topics covered by the Brazilian literature on AI in distance 
education? 

(3) What are the theoretical bases used in Brazilian studies on AI in distance education? 

(4) Which results do Brazilian research suggest on the use of AI in distance education? 


Understanding how Brazilian researchers are addressing this theme is important 
to identify the trends, gaps, and opportunities of research in distance education. Given the 
relevance of the Brazilian education market for Latin America, research evidence may be useful 
for several other countries that face similar issues regarding their educational structure. The 
development and implementation of educational technologies should be based on scientific 
evidence to introduce best practices to the learning process. Otherwise, when the academy is 
not utilizing technologies that have been implemented in educational practice, it can lead to 
inefficacy and inequality in the learning process. 


Research Methodology 
General Background 


The research aimed to identify the current state of Brazilian literature on AI applied to 
distance education. In this sense, it is characterized as exploratory because it deals with a topic 
still little explored by Brazilian researchers in the field of Education. It is also classified as 
bibliographic research, as it performs an SLR to answer the proposed research question. Finally, 
the data were analyzed using qualitative strategies to interpret the results and quantitative 
metrics to present the findings. 


Procedures 


The SLR conducted in this study used the five steps indicated by Klun et al. (2003). To 
know: 1) framing questions for a review, ii) identifying relevant work, ii1) assessing the quality 
of studies, iv) summarizing the evidence, and v) interpreting the findings. For the first step, the 
questions formulated should enable achieving the objective stated, which was to identify the 
current state of Brazilian literature on AI applied to distance education. As mentioned in the 
introduction topic, four questions were addressed to achieve the paper’s objective. 
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To cover the second step, it was used the database named Brazilian Digital Library of 
Dissertations and Theses (known as BDTD in Portuguese), which contains all of the master’s 
theses and doctoral dissertations of the most important Brazilian HEIs. The reason for using 
theses and dissertations is due to the research question that guides this study. In this sense, 
through the review of theses and dissertations produced in Brazilian graduate programs, it is 
possible to identify the solidity of the research on the subject. By analyzing the theses and 
dissertations on this subject it is possible to identify research groups and projects that are 
developed in graduate programs, which reveal the state of research on AI that is applied to 
distance education in Brazil. 


Data Collection and Analysis 


Using the BDTD database (https://bdtd.ibict.br/vufind/), it was applied a protocol 
of research that was composed of the combination of the following codes of search (all in 
the Portuguese language): “artificial intelligence” with “teaching”, “distance education”, 
“education”, and “learning”. The codes were used for the titles, abstracts, and keywords of 
theses and dissertations registered in the BDTD. By this protocol, it was possible to find the 
first amount of 519 studies. Table 1 shows the results of this search. In this first sample, 241 
studies appeared more than once. After excluding the repeated research, it was analyzed the 
content of each study and applied the excluding protocol. With this process, it was possible to 
select a final sample of 63 studies to conduct the SLR. Of this total, 26 are master’s theses and 
37 doctoral dissertations. 


Table 1 
Results of Theses and Dissertations Collected on BDTD 


Research Data 


Codes : Total 
Keywords Title Abstract 

“Artificial Intelligence” & “Teaching” 0 0 4 4 

“Artificial Intelligence” & “Distance Education” 0 0 30 30 

“Artificial Intelligence” & “Education” 51 2 154 207 

“Artificial Intelligence” & “Learning” 54 4 220 278 

Total 105 6 408 519 


The excluding protocol was processed as follows: 


(a) 105 studies were discarded because they were related to the use of AI at some level 
of education but not in the context of distance education. 

(b) 60 theses and dissertations collected dealt with AI but were not directly applicable to 
teaching at any level or modality. 

(c) 14 studies analyzed were not directly related to AI and were also excluded from the 
final study sample. 

(d) 36 studies were not available for consultation on the platform of their respective 
graduate programs when the analysis was conducted, either due to a link failure or 
because the file was unavailable for consultation. 


The data collection, which represents the second step of our SLR, was done between 
March 2021 and June 2021. 

For the third step of SLR, it was analyzed the format of the studies collected. The topic 
covered by this SLR is multidisciplinary, so it was common to find a different theoretical basis 
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and methodological procedures. Despite this diversity, it was possible to consider all of the 
results found well supported by the research conducted. In this sense, it was not necessary to 
discard any more research. 

The fourth step was covered by qualitative analysis, as such a codification process. In this 
stage, the main parts of the theses and dissertations were read, making it possible to understand 
the topic as it related to the results. The codification was conducted using the themes identified 
by Zawacki-Richter et al. (2019). Furthermore, throughout the process, three new codes were 
added, as the themes that emerged from the data were not found in the previous research. 

Finally, the fifth and last step of the SLR, shown in the last section of this paper, provides 
our conclusion about the stage of Brazilian literature in AI, identifying the maturity of the 
research in this field, highlighting the gaps existing, and suggesting trends for future research. 


Research Results 


The Programs that the Studies Came from 


Figure 1 shows the distribution of 63 theses and dissertations collected in the SLR by 
the year of publishing. The first study analyzed was published in 1994 and the last in 2020. 
In almost this entire period at least one study was published on AI in Brazilian HEIs. The 
exceptions were 1995, 1996, 1998, 2012, and 2015. Figure | also indicates that, from 2001 to 
2004, it was the period with more studies on AI, representing 26 occurrences (41%). Despite 
said evidence, the last four years analyzed (2017-2020) also showed an improvement in the 
number of theses and dissertations on AI with 13 occurrences (21%). 


Figure 1 
Number of Studies Collected in SLR by the Year of Publishing 
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Table 2 shows the HEIs in which the selected studies were developed. It also presents 
the type of graduate program that the theses and dissertations were carried out. The first 
evidence it brings is the relevance of Brazilian public HEIs to the research developed on AI. 
From 63 theses and dissertations, only three studies were carried out at private institutions 
(International University Center - UNINTER, University of Vale do Rio dos Sinos —-UNISINOS, 
and Presbyterian Mackenzie University - UPM). All of the other studies came from public 
institutions, that is, organizations maintained by the Brazilian government. 
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Table 2 also shows that the two HEIs with the most research selected were the Federal 
University of Rio Grande do Sul (UFRGS) and Federal University of Santa Catarina (UFSC) 
with 15 studies each. The following is the Federal University of Campina Grande (UFCG) with 
nine studies. The fourth is the State University of Campinas (Unicamp), with seven selected 
studies. The next three HEIs appeared with two occurrences, which are the Federal University 
of Amazonas (UFAM), Federal University of Pernambuco (UFPE), and Federal University of 
Pelotas (UFPEL). In total, 18 HEIs had at least one thesis/dissertation on AI applied to distance 
education. However, 11 HEIs appeared with only one study, which indicates that this subject 
may not be a strong research topic for these institutions. 


Table 2 
Programs in Which the Studies Were Developed 


Absolute 


HEI Frequency Graduate Program 

UFRGS 15 Computer Science (12) / Multidisciplinary (3) 

UFSC 15 Engineering (12) / Computer Science (3) 

UFCG 9 Computer Science (9) 

Unicamp 7 Computer Science (3) / aa Ge Rr Language and Literature (1) / 
UFAM 2 Computer Science (2) 

UFPE 2 Computer Science (2) 

UFPEL 2 Portuguese Language and Literature (1) / Computer Science (1) 
UFAL 1 Computer Science (1) 

UFMA 1 Engineering (1) 

UFPA 1 Engineering (1) 

UFPB 1 Education (1) 

UFRN 1 Computer Science (1) 

UFV 1 Computer Science (1) 

UFVJM 1 Education (1) 

UnB 1 Arts (1) 

UNINTER 1 Education (1) 

Unisinos 1 Portuguese Language and Literature (1) 

UPM 1 Engineering (1) 


Note. Number of occurrences in parentheses 
In addition, Table 2 also shows that the theses and dissertations published in Brazilian 


graduate programs came from diverse backgrounds. This is important evidence, as the AI 
subject requires a multidisciplinary perspective. However, it was also possible to identify a 
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predominance of Computer Science programs with 38 studies (56%), followed by Engineering 
with 17 studies (27%). Theses and dissertations from Education programs represented only four 
occurrences (6%). 


The Topics Covered by the Selected Studies 


Table 3 shows the topic covered by the studies selected in the SLR. It was possible 
to classify all the research into seven codes: (i) Intelligent Tutoring Systems, (ii) Adaptive 
Systems and Personalization, (iii) Assessment and Evaluation, (iv) Profiling and Prediction, 
(v) Development and Management of Educational Content, (vi) Virtual Learning Environment, 
and (vii) Theoretical and Empirical Basis for Educational Software Development. 


Table 3 
Topics Covered by Selected Studies 


Relative 


Topics Covered Absolute Frequency Frequency (%) 
Intelligent Tutoring Systems 28 44 
Adaptive Systems and Personalization 11 18 
Assessment and Evaluation 9 14 
Profiling and Prediction 5 8 
Development and Management of Educational Content 5 8 
Virtual Learning Environment 3 5 
Theoretical and Empirical Basis for Educational Software Development 2 3 

Total 63 100 


The most frequent topic in the selected Brazilian theses and dissertations was the 
“Intelligent Tutoring Systems”, with 28 occurrences (44%). In this type of research, it was 
possible to find AI propositions and assessments created to help teachers and students in the 
process of teaching-learning. The second most frequent topic, in turn, was “Adaptive Systems 
and Personalization”, with 11 studies (18%). In this type of research, it was common to present 
solutions to help students with their own needs in the educational process. The third most 
common topic in the Brazilian theses and dissertations on AI applied to distance education was 
“Assessment and Evaluation”, with 10 cases (16%). These studies usually presented solutions 
to improve strategies for valuing students’ achievement. 

Together, these three first topics represented an amount of 76% of selected studies 
in this SLR. Therefore, despite the other four occurrences, it was possible to notice that the 
Brazilian literature on AI applied to distance education selected in this SLR was concentrated 
on only a few themes. The “Profiling and Prediction” and “Development and Management of 
Educational Content” represented each 8% of the occurrences. Finally, the last two topics were 
“Virtual Learning Environment” with five studies (5%) and “Theoretical and Empirical Basis 
for Educational Software Development” with only two cases (3%). 
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The Theoretical Bases Used in Selected Studies 


Table 4 shows the educational theories most used by the theses and dissertations selected 
to support the phenomenon studied. The result indicated that the three highlighted theories are 
centered on students’ learning process. 


Table 4 
Educational Theories Commonly Used in Selected Studies 


Theories Brief Explanation Authors 


It elucidates the different aspects of student's learning process, 


beaming etyies assuming that different individuals have different learning styles Renal ihols 
Self-Regulated It explains how students self-manage their thoughts, feelings, , 

; ‘ . : bee Barry Zimmerman 
Learning and actions to achieve their personal goals and objectives 


Cognitive Learning Explains the mental process related to the learning process and Jean Piaget, Lev Vygotsky, 
Theories shows how it is influenced by internal and external factors and Reuven Feuerstein 


Although many studies from different areas have used important educational theories, 
it was possible to identify theses and dissertations that did not support their research from the 
perspective of education. In addition, it was possible to perceive that little attention is given 
to the teachers’ perspectives on the use of AI. As many studies focused on the implementation 
of AI to (direct or indirectly) improve the teachers’ work, it is important to understand how 
AI changes the teaching process. A critical theoretical perspective on the use of AI was also 
neglected in the studies selected. The use of technologies in the educational process might 
cause negative impacts on the society that must also be investigated through a critical lens, 
highlighting points that go beyond improving student learning. 


Exemplification of Research on the Use of AI in Distance Education 


As highlighted before, this SLR found that a large part of the theses and dissertations 
reviewed could be classified as related to the topic of “Intelligent Tutoring Systems” (44% of the 
total). Although the subject investigated in this area is diverse, it was possible to identify that, 
in this group, the methodology usually employed by authors was related to the development 
and/or testing of new technologies powered by an AI solution. Intelligent tutoring systems can 
be implemented at the educational level for many users: students, teachers, educational staff, 
and administrators. In this SLR, almost all 28 studies analyzed from the “Intelligent Tutoring 
Systems” group were applied at the student level. 

An example of research collected in this SLR and classified in this group was the 
doctoral dissertation of Jaques (2004), developed in the Computer Science program at the 
Federal University of Rio Grande do Sul. This study proposed an intelligent tutoring system 
to support students affectively. The mediator agent developed in this study acted according to 
students’ emotions, which was structured based on cognitive learning theories (see Table 4). To 
do this, the system proposed used events that had occurred in the virtual environment, such as 
the type of answer given by students (right or wrong) and whether the student was up to date or 
not with the activities or content of the course. Despite the proposition of this mediator agent, 
this research did not evaluate the effectiveness of the AI proposed. 

Another example of research classified in the “Intelligent Tutoring Systems” group in 
this SLR was the master’s thesis carried out in the Portuguese Language and Literature program 
by Moro (2019) which analyzed the adequacy of a software program, named Skylar, which 
automatically transcribes and translates speech. Skylar was used in a video class for the MBA 
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in Agrobusiness from the University of Sao Paulo, which used the format of distance education. 
To allow the internationalization of the program, experts from different countries were invited 
to participate in the classes. In this sense, AI such as Skylar becomes important to help students 
with gaps in international languages to understand what is being said during classes. Using a 
mixed methods approach, this research found that the AI analyzed was positive in expressing 
what the speaker said. Nevertheless, the system showed gaps when the context of the sentences 
was analyzed. 

Even though the majority of studies from the “Intelligent Tutoring Systems” group in 
this study were applied to the student level, it is necessary to highlight that one study was 
conducted at the administrative level. This was the case of the dissertation of Lucena (2018), 
which was carried out in a Computer Science program. It studied an intelligent agent built 
to manage distance learning courses based on previous problems and solutions detected in 
distance education courses at the Federal University of Amazonas. After the development of 
the system (called iDE) by the researcher, using the framework of }COLIBRI 2, it was validated 
by ten specialists in distance education. The results of this validation showed that the proposed 
technological solution was positively evaluated and considered relevant for the management of 
distance courses. 

The second most frequent topic of this SLR was the “Adaptive Systems and 
Personalization” with 18% of all of the cases. An example of study from this group was the 
master’s thesis of Ferreira (2018), carried out in the Computer Science program at the Federal 
University of Pernambuco. This research developed and tested an AI solution to recommend 
learning activities to students from distance education courses based on autoregulated patterns 
available in the virtual environment. In this sense, the software proposed allowed for: (1) 
analyzing the students’ performance; (ii) extracting the behavioral characteristics that influence 
their performance, and (iii) recommending actions to improve it. This AI was tested with more 
than 30,000 students from a Brazilian university and the results showed that the technological 
solution was capable of properly capturing the students’ learning styles and their behavioral 
characteristics. With this good fit, the software was efficient in recommending additional 
training (texts and/or videos) to students according to their performance. 

The third most recurrent topic that emerged from this SLR was “Assessment and 
Evaluation” with 16% of all the research. Among this group was Riedo (2020), a dissertation 
from the Education program at the State University of Campinas. It is important to remember 
that only 6% of theses and dissertations selected in the present SLR came from Education 
programs, despite the importance of the phenomenon analyzed in this area. In this sense, the 
aforementioned dissertation aimed to develop and test an AI program able to evaluate students’ 
texts in the context of distance education, given to participants’ immediate feedback. To test 
the AI solution proposed in this research, the author used data from three courses offered by 
the State University of Campinas in the Massive Open Online Course (MOOC) model. Using 
a total of 8,788 essays, the study was able to confirm the ability to qualitatively differentiate 
the analyzed texts, indicating that this AI solution had wide applicability in distance education. 

In general, the theses and dissertations analyzed in this study confirmed the ability of AI 
to transform the distance education system, despite some points of attention. More importantly, 
these studies explicitly detailed the current presence of AI in the context of distance education. 
In addition, since the most used methodology in the selected studies consisted of carrying out 
tests for Al-based solutions, it was possible to predict some impacts of these technologies on 
the educational process. 
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Discussion 


The introduction of AI in the education field has the potential to transform the way society 
learns (Bregant & Abersek, 2011). As in many other areas, AI solutions in the educational 
environment can transform jobs and impact teaching activities. In this sense, the SLR carried 
out in this research makes it clear that AI is already an alternative to improve the distance 
learning environment in Brazil since many theses and dissertations from Brazilian graduate 
programs develop and evaluate technologies powered by AI. 

However, corroborating with previous research (Zawacki-Richter et al., 2019) despite 
the relevance of this topic for the present, most Brazilian theses and dissertations on AI 
applied to distance education came from Computer Science and Engineering. This evidence 
is concerning as it shows that the Education field is not as engaged with the subject related to 
the application of AI in distance education as other areas of knowledge and this phenomenon 
may have some roots. Despite the expressive growth of the distance education system in Brazil 
in the last decade, and especially during the Covid-19 pandemic, this modality still presents 
resistance within the field of Education (Machisotti, 2022). This finding may also reflect the 
teachers’ fear that technological solutions might replace them, making AI research undesirable 
in the Education graduate programs. 

The timid participation of Education graduate programs in this topic can have 
consequences for the field. First, it can delay the improvements AI can generate to the learning 
process in distance education. It is important to note that this modality of education has the 
potential to reach individuals who have no other possibility of studying, which in a continental 
country like Brazil, it is a significant portion of the population. In this sense, the delay in 
improving the quality of distance learning has direct economic and social impacts on society. 
Additionally, the absence of the Education area in the discussion of AI applied to distance 
education can also make technological solutions developed in other fields (like Computer 
Science and Engineering) pedagogically fragile (AberSek & Kordigel-AberSek, 2011). 

Additionally, it was also possible to identify that the topics covered by Brazilian research 
on AJ are concentrated on just a few themes. Just three topics represent an amount of 76% of 
selected studies in this SLR. Many of the studies reviewed, especially from Computer Science, 
propose and analyze anew educational innovation. Technology represents ademand from society 
to solve the problems that people face in the environment (Pinto, 2005). With the development 
of the Internet, computer, and other technologies, new demands are made by society, including 
for distance education (Chen et al., 2020), and AI has been considered a key tool to solve these 
issues (Ertel, 2018). In this sense, the attention paid by the Brazilian researchers to restricted 
topics may limit the capability of innovation of AI applied to distance education. The greater 
the diversity of themes on AI, the more likely it will be that new disruptive technologies can be 
implemented in distance education. 

Another important evidence of this SLR is the predominance of the “Adaptive Systems 
and Personalization” topic in Brazilian theses and dissertations. As highlighted by Zawacki- 
Richter et al. (2019), there is no agreement in the literature on the definition of an adaptative 
system and sometimes it is coincident with intelligent systems. Although the studies classified 
in this group by this SLR also showed many approaches, all of the research collected was related 
to the possibility of providing individualized education through AI solutions, which makes this 
kind of investigation a specific topic of research. Adaptative systems for the personalization of 
the learning process have been considered important to all educational levels, but especially to 
distance education due to its heterogeneity (Maghsudi et al., 2021). For developing countries 
like Brazil, which still have large socioeconomic disparities, personalized education has the 
potential to improve the learning environment. 
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Through this SLR, it is possible to suggest some trends for future research, considering 
the international agenda on AI applied to distance education. The first one is the need to better 
understand the impact of AI on changes in teachers’ work. Many new technologies already 
implemented in the educational market are changing the teaching process, demanding teachers 
to learn new skills and reducing their job opportunities in the market. A critical approach may 
be needed for a deep understanding of these changes, including the ethical implications of using 
AI in distance education. 


Conclusion and Implications 


This research aimed to identify the state of Brazilian literature on AI applied to distance 
education. In this sense, an SLR was carried out using the BDTD database, which contains theses 
and dissertations from Brazilian HEIs. Following the five steps suggested for an SLR, it was 
possible to select 26 master’s theses and 37 doctoral dissertations (63 studies in total). Through 
this procedure, it was possible to reach some conclusions and point out some implications. 

The first evidence is that studies on AI applied to distance education in Brazil come 
mostly from different areas of knowledge, such as Computer Science and Engineering. Only 
6% of the selected studies in this SLR came from the Education field. This might represent 
a significant gap related to research in distance education of Brazilian Education graduate 
programs, as AI represents an important instrument already used in virtual courses in Brazil. 

Other important evidence of this SLR is that the production of the theses and dissertations 
on AI applied to distance education in Brazil is mainly concentrated on a few HEIs, all of 
which are public institutions. This also represents a risk regarding the development of the 
topic since AI implementation should be encouraged from a multidisciplinary perspective. The 
more concentrated it is, the less chance there is of innovation. Therefore, the most common 
themes were also focused on three topics: “Intelligent Tutoring Systems”, “Adaptive Systems 
and Personalization”, and “Assessment and Evaluation”. The concentration of themes is also 
a risk for the literature on AI applied to distance education since many problems already faced 
by society with the introduction of technological innovation may not be investigated in this 
research. 

Finally, the SLR shows that the theoretical approach of many selected studies is flawed. 
Despite the use of some important frameworks that explain the learning process from students’ 
point of view, little is discussed about teachers’ interaction with AI in distance education. This 
is also important, as almost all of the selected theses and dissertations propose and analyze new 
technological solutions that will impact the teaching process and, possibly, the teaching career. 

It is important that the Education area becomes more familiar with the topic of AI 
applied to online education. Only a few works collected in this SLR came from a program in 
Education in Brazil, which is evidence that agrees with the findings of international literature. 
For studies developed in areas other than Education, it is important to improve the theoretical 
framework, deepening theories that explain the whole phenomenon that affects the teaching- 
learning process in distance education. 

Although this SLR analyzed the Brazilian context, its results are in line with the previous 
research carried out in different contexts. In this sense these findings can also contribute to 
the development of international research, adding the Brazilian evidence to the international 
literature on AI applied to distance education. 

Additionally, Brazil has some specificities that make its educational system an important 
case to analyze. First, this country represents the biggest educational market in Latin America, 
with many individuals still without access to an HEI, which is also the reality of many developing 
countries. Moreover, in the Brazilian private educational market, there has been a concentration 
in the last years with a few private higher education companies controlling a significant part of 
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the distance education market. In this sense, the research developed in the Brazilian graduating 
programs reflects these complexities, searching for solutions for the problems researchers 
perceive in their educational context. 
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Abstract 


Determining students' perceptions of problem-solving is an important step to improve their mathematical 
problem-solving skills. For this reason, the aim of this study was to develop a scale to determine secondary 
school students’ perceptions of mathematical problem solving. In the study conducted in a basic research 
model, a scale application, one of the quantitative research methods, was used. The sample of the study 
consisted of 325 secondary school students. Validity and reliability analyses were made with the data 
obtained on the sample. Five sub-dimensions of the scale were determined through exploratory factor 
analysis. Confirmatory factor analyses were performed for these 5 sub-dimensions, and the fit indices 
were found to range from acceptable to excellent. The explained variance of the “Mathematical Problem- 
Solving Perception Scale” was 66.15%, and the Cronbach alpha value for the whole scale was .93. It 
was concluded that the scale would reveal the perception levels of secondary school students towards 
mathematical problem solving in a valid and reliable way. 

Keywords: mathematics, perception, problem solving, scale development, secondary school students 


Introduction 


Besides the ability to acquire and learn mathematical knowledge, it is of great importance 
for the individual to be able to use this mathematical knowledge in solving various problems 
he/she encounters. In the 21st century, with the development of technologies for accessing 
information, it has become easier to obtain information and the need for information to be readily 
available to the individual has decreased. Instead, the importance of determining solutions for 
problems that are encountered and of putting these solutions into practice has increased. In 
addition, nowadays, in accordance with the philosophy of progressive education (Dewey, 1997) 
arising from the philosophical movement of pragmatism, which affects education systems, the 
view is widespread that the individual’s possession of information that he/she will rarely or 
never use throughout his/her life will not provide him/her with any benefit. According to this 
view, it is considered important to develop the ability to cope with situations and problems 
that an individual may encounter throughout his/her life. Therefore, it is of great importance 
that individuals are equipped with problem-posing and problem-solving skills in the education 
process. 

Rather than raising individuals who accept ready-made knowledge by memorizing it, 
knowledge societies aim to raise individuals who know how to put learning into practice, what 
and how to learn and where to use this learning, and who can produce new knowledge based 
on this information (Giiven & Kiiriim, 2008). Accordingly, in today’s knowledge societies, 
problem-solving skills are among the most important skills that individuals need to acquire in 
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their educational life. Problem-solving skills have an important place not only for the education 
lives of individuals, but also for their whole lives. By using problem-solving skills, individuals 
can overcome all kinds of problems that they may encounter during their lives. 

A problem is the conflict that an individual experiences when he/she is prevented from 
achieving a goal (Morgan, 1995). Olkun and Toluk (2004) define problems as situations which 
awaken the desire in a person to solve them, but which, because the solution procedure is not 
ready, the person can solve by using his/her knowledge and experience. For an individual to 
solve a problem that he/she encounters, he/she should be able to use basic problem-solving 
skills such as the ability to establish relationships and differences between concepts, to think 
algorithmically, and to analyse and approach the problem from different perspectives. These 
skills constitute the individual’s problem-solving skills as a whole. Therefore, one of the main 
goals of education and training programmes is to raise individuals who can keep pace with the 
age they live in and solve the problems they encounter in a beneficial way. Accordingly, it can 
be said that organising curriculum development efforts in a way that focuses on the objectives 
for problem-solving skills will contribute to the development of the individual and society. 

The general trend in mathematics teaching in Tiirkiye in recent years has been to give 
weight to problem-solving and problem-posing activities, and to frequently use problem- 
based instruction and cooperative teaching approaches (Ministry of National Education 
[MoNE], 2022). Therefore, it can be said that problem posing is not the next step after problem 
solving, but that there is a spiral relationship between them rather than a hierarchical one. 
The implementation of problem-solving and problem-posing activities in mathematics teaching 
within the framework of this spirality can make the process more efficient and effective. 

In mathematics teaching, the individual’s analysis of events within the framework of 
logic, establishment of cause-effect relationships, and determination and implementation of the 
necessary process steps are among the learning outcomes of the mathematics curriculum (MoNE, 
2022). Since these stated outcomes are directly addressed in problem-solving instruction, 
it can be said that it is important to include studies that focus on problem-solving skills in 
mathematics teaching in the literature. The goal of contemporary education systems should be 
to raise individuals with advanced problem-solving skills (Bayazit & Aksoy, 2010). Therefore, 
it is important for the individual to be trained in order to produce knowledge throughout his/ 
her education life and to find solutions to problems he/she will encounter by producing new 
knowledge. 

The skills of reasoning, estimation, analysis, synthesis, and evaluation, which are 
metacognitive skills, are target behaviours included in problem-solving and problem-posing 
instruction. It is possible to find a solution to a commonly encountered mathematical problem 
by associating it with a problem with previously known solutions. However, the individual’s 
ability to find a solution to a problem unlike any that he/she has encountered before can be 
achieved by applying the steps of separating the components of that problem, making a plan, 
implementing the plan appropriately and describing what he/she has achieved. It is possible 
to foster the metacognitive skills included in these steps by teaching mathematical problem 
solving and problem posing. 

Students’ perceptions of problem solving have an active place in any kind of study 
to be made for developing students’ mathematical problem-solving skills. For this reason, it 
is important for studies to be made to determine the level of students’ perceptions towards 
problem solving. In the literature, there are measurement tools that measure students' attitudes 
towards mathematics (Eskici et al., 2017; Giilburnu & Yildirim, 2021; Tezer & Ozcan, 2015), 
their attitudes towards problem solving in mathematics lessons (Aydin, 2020), and their 
perceptions towards social problem solving (Hawkins et al., 2009). However, there is no scale 
in the literature that aims to determine students' problem-solving perceptions in mathematics 
lessons. Therefore, it can be said that the scale developed in this study has unique value for 
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the literature. In addition, it can be said that the reason for the limited number of studies in the 
literature that investigate students’ mathematical problem-solving perceptions (Ersoy, 2022) is 
due to the absence of a measurement tool that reveals secondary school students’ perceptions of 
problem solving. In line with this need, the “Mathematical Problem-Solving Perception Scale” 
developed by the researchers aims to determine the mathematical problem-solving perceptions 
of secondary school students. 


Research Methodology 


Model 


This study, whose aim was to develop a valid and reliable measurement tool that will 
determine secondary school students’ perceptions of mathematical problem solving, was 
prepared in a basic research model. In the research, 5-point Likert-type scale development, 
validity and reliability studies were carried out. 


Population and Sample 


In order to ensure the construct validity of the perception scale consisting of 33 items, 
which was prepared by the researcher and whose content validity was ensured by obtaining 
expert opinion, a pilot study was conducted. A total of 2412 secondary school students studying 
in Antalya, Tiirkiye, made up the population of the study. Due to the COVID-19 measures, 
the pilot implementation was carried out in the online environment because the students could 
not be found in face-to-face education. In all, 325 secondary school students were included 
in the pilot study with the simple random sampling method. According to Balci (2018), in 
the simple random sampling method, it is independently and equally likely that each sub-unit 
of the population with which the study is conducted will be the research sample. With the 
data obtained, exploratory factor analyses (EFA) and confirmatory factor analyses (CFA) were 
performed within the scope of the construct validity studies. 

The sample consisted of 325 secondary school students who participated in the pilot 
implementation in the specified population of the study in the fall semester of 2021-2022. The 
demographic information of the secondary school students included in the sample is given in 
Table 1. 


Table 1 
Information on the Sample of the Pilot Implementation Study for the Mathematical Problem- 
Solving Perception Scale 


Number (f) Percentage (%) 
Female 180 55 
Gender Male 145 45 
5" Grade 50 16 
6" Grade 77 24 
Grade 7" Grade 99 30 
8" Grade 99 30 
General Secondary School 291 89 
imam Hatip Secondary 11 4 
Type of Secondary School shag} 
Attended 
Regional Boarding 3 1 
Secondary School 
Private Secondary School 20 6 
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As seen in Table 1, among the secondary school students included in the research sample, 
180 (55%) were girls and 145 (45%) were boys. Fifty (16%) of the students were in 5th grade, 
77 (24%) were in 6th grade, 99 (30%) were in 7th grade, and 99 (30%) were in 8th grade. 
Among the types of schools they attended, 291 (89%) were general secondary schools, 11 (4%) 
were imam hatip secondary schools, 3 (1%) were regional boarding secondary schools, and 20 
(6%) were private secondary schools. 


Measurement Tool 


In this study, a “Mathematical Problem-Solving Perception Scale” was developed 
for secondary school students. In the study, the process of seeking answers to the question 
“What are secondary school students’ perceptions about learning problem-solving skills in 
mathematics?” was begun with a literature review. As a result of the literature review, the item 
pool was discussed within the scope of the following sub-dimensions: Students’ perceptions of 
willingness and voluntariness, perceptions of fear and hesitation, and perceptions of importance 
and value related to mathematical problem-solving skills. 

In the sub-dimensions stated above, the scale items were created for the feeling of 
happiness, the feeling of fun and pleasure, the feeling of insistence and determination for success, 
the feeling of sadness and distress, the feeling of interest, the feeling of fear and hesitation 
(avoidance), the feeling of giving up, and the feeling of pride. Opinions were obtained about 
the appropriateness of the prepared items by two experts from the department of curricula and 
instruction and by one expert from the department of measurement and evaluation in education. 
In line with these opinions, the scale items were finalised. The “Mathematical Problem-Solving 
Perception Scale” includes statements graded on a 5-point Likert-type scale in the form of (1) 
“Strongly disagree’, (2) “Disagree”, (3) “Undecided”, (4) “Agree”, and (5) “Strongly Agree”. 


Data Collection Tool 


The pilot implementation of the “Mathematical Problem-Solving Perception Scale” was 
made online via a website opened by the researcher. After obtaining the necessary permission 
from the Antalya Provincial Directorate of National Education, students who volunteered to 
respond to the scale were enabled to complete the scale online via the specified website. 


Data Analysis 


For data reliability, the results of the Cronbach alpha test, and for EFA, the results of 
the Bartlett test and Kaiser-Meyer-Olkin (KMO) test were examined. In the EFA, items with 
overlapping factor loadings were removed and Varimax rotation analyses were repeated. CFA 
was performed on 24 items and 5 sub-dimensions obtained as a result of the EFA and fitted 
t-values were obtained. These analyses were carried out with the SPSS 23.0 and LISREL 8.8 
software programs. 


Research Results 
Model 
Exploratory factor analysis (EFA): Prior to the EFA, which was carried out to ensure the 
construct validity of the problem-solving perception scale, the Bartlett test of sphericity was 


performed, and the KMO value was found to determine the suitability of the data obtained from 
the sample. As a result of the analysis, it was determined that the KMO value (0.905) was greater 
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than 0.60, and it was concluded that the data were suitable for factorisation (Biiyiik6ztiirk, 
2018). 

When the factor loadings obtained with the varimax rotation technique were examined, it 
was determined that the factor loadings were above 0.40, and that there were 7 items with high 
loading values in more than one factor and the difference between these values was below 0.1. 
After assessing whether these items would cause a significant loss in terms of content validity 
of the scale, the items, which were determined to be overlapping, were removed from the scale. 
After the determined items were removed from the scale, EFA was repeated on the remaining 
24 items. As a result of the analysis, a five-factor structure with an eigenvalue greater than | and 
representing the theoretical structure was obtained. The data obtained as a result of the factor 
rotation are given in Table 2. 


Table 2 
Final Factor Rotation Results 


Items Factor-1 Factor-2 Factor-3 Factor-4 Factor-5 

Item-19 831 

Item-28 830 

Item-31 799 

Item-33 762 

Item-8 145 

Item-18 141 

Item-15 721 

Item-27 706 

Item-23 .650 

Item-24 143 

Item-20 701 

Item-22 675 

Item-25 .658 

Item-16 604 

Item-21 152 

Item-10 123 
2 113 

Item-11 685 

Item-1 827 

Item-2 820 

Item-6 621 

Item-3 091 

Item-26 827 

Item-13 157 
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When the factor rotation results given in Table 2 are examined, it can be seen that items 
19, 28, 31, 33, 8, 18, 15, 27 and 23 are in the Ist factor, items 24, 20, 22, 25 and 16 are in the 
2nd factor, items 21, 10, 12 and 11 are in the 3rd factor, items 1, 2, 6 and 3 are in the 4th factor, 
and items 26 and 13 are in the 5th factor. In the literature, there is a common view that the 
number of items in the dimensions should be at least two (Cokluk et al., 2012). Therefore, it 
was determined that the number of items in the factors included in the scale was appropriate. 
Among the items divided into sub-dimensions as a result of the EFA, items 8, 13, 15, 18, 19, 
23, 26, 27, 28, 31 and 33 contain statements expressing negative perceptions. The other scale 
items refer to positive perceptions. 

The sub-dimensions of the scale are named: (1) Fear and hesitation towards problem 
solving, and becoming distressed in problem-solving activities, (2) Showing interest in problem 
solving in class and feeling proud because of this, (3) Interest in problem-solving activities 
outside of class and feeling happy because of this, (4) Willingness for problem-solving 
activities in class and feeling happy because of this, and (5) Giving importance to problem- 
solving activities. 

Confirmatory factor analysis (CFA): CFA were conducted to check the construct validity 
of the 5 sub-factors obtained as a result of the EFA of the problem-solving perception scale. 
CFA is a type of analysis that should be used in the scale development process to examine the 
latent structure of a measurement tool (Brown, 2006). 

Using the data collected from the study group, first-order CFA was performed to test 
whether the structure consisting of 24 items and five factors obtained from the EFA was 
confirmed. Regarding the obtained fit indices, the generally accepted “excellent fit criteria” and 
“acceptable fit criteria” regarding goodness-of-fit in the literature (Kline, 2011; Schumacker & 
Lomax, 2010; Tanaka & Huba, 1985) were taken into account. The fit index values obtained 
from the CFA and the results that emerged in this direction are given comparatively in Table 3. 


Table 3 
Fit Indices Obtained from CFA 


Fit Index Excellent Fit Measures Acceptable Fit Measures Measure Result 
y2ldf Os y2ldfs 2 2s y2ldf<5 3.12 Acceptable 
GFI .95 < GFI < 1.00 .85 $ GFI S$ 95 85 Acceptable 
CFI .97 S$ CFI S$ 1.00 95 $ CFI < .97 95 Acceptable 
NFI .95 < NFI < 1.00 90 <NFI< .95 93 Acceptable 
NNFI .97 = NNFI $1.00 .95 < NNFI $ .97 95 Acceptable 
IFI .95 < IFI $ 1.00 90 SIFI Ss .95 95 Excellent 
RMSEA 00 < RMSEA § .05 05 < RMSEA S .10 081 Acceptable 
SRMR .00 < SRMR s .05 .05 $ SRMR s .10 .065 Acceptable 
AGFI .95 $ AGFI $ 1.00 .50 $AGFI $ .95 68 Acceptable 


When the fit indices obtained as a result of the CFA in the study are examined, it can 
be seen that they are: y2/df = 3.12 (2 < y2/df < 5) “Acceptable”, GFI = .85 (.85 < GFI < 
.95) “Acceptable”, CFI = .95 (.95 < CFI < .97) “Acceptable”, NFI = .93 (.90 < NFI < .95) 
“Acceptable”, NNFI = .95 (.95 < NNFI < .97) “Acceptable”, IFI = .95 (.90 < IFI < .95) 
“Excellent”, RMSEA = .081 (.05 < RMSEA < .10) “Acceptable”, SRMR = .065 (.05 < SRMR 
<.10) “Acceptable”, and AGFI = .68 (.50 < PGFI < .95) “Acceptable”. One of the fit criteria not 
included in the table is the “critical N” value, in which the adequacy of the research sample is 
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evaluated. In SEM, a sample size of 200 or more is accepted as an indicator of adequate model 
fit in terms of the critical N statistic (Bollen, 1989). Therefore, it can be said that the sample size 
of 325 used in the research was adequate (Celik & Yilmaz, 2016). 

When the RMSEA descriptive fit index, which is based on the difference between the 
estimated covariance matrix for the model and the sample covariance matrix, was evaluated, 
it was determined that the model showed an acceptable fit. When the other fit criteria are 
considered, it is seen that the model is significant as a whole. The factors of the CFA model, the 
factor loads, and the t-values of the items are given in Table 4. 


Table 4 
Factors, Factor Loads, and t-values of Items in the CFA Model 


Factor-1 Factor-2 Factor-3 Factor-4 Factor-5 
Item CFA CFA CFA CFA CFA t-value 
8 1.08 14.50 
15 1.06 15.40 
18 1.04 14.53 
19 1.18 16.84 
23 1.01 13.89 
27 1.04 14.69 
28 1.13 17.33 
31 1.18 16.71 
33 1.18 16.16 
16 74 12.01 
20 83 11.66 
22 87 14.59 
24 80 12.51 
25 .99 16.02 
10 1.02 19.33 
11 1.07 19.42 
12 81 18.36 
21 81 12.65 
1 99 20.35 
2 1.00 20.27 
3 12 12.13 
6 09 10.51 
13 1.13 8.70 
26 19 7.28 


As can be seen in Table 4, “it is significant at the level of 0.05 if the t-value exceeds 1.96, 
and at the level of 0.01 if it exceeds 2.56” (Schumacker & Lomax, 2010). Meaningless ones 
should be removed from the scale. 
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Reliability 
The item-total correlations of the items in the scale are calculated in Table 5. 


Table 5 
Item-Total Correlation Coefficients 


Item Item-Total Correlation Item Item-Total Correlation 
Item-1 629 Item-19 647 
Item-2 .603 Item-20 516 
Item-3 629 Item-21 574 
Item-6 .306 Item-22 .665 
Item-8 AT9 Item-23 415 
Item-10 693 Item-24 451 
Item-11 664 Item-25 692 
Item-12 O71 Item-26 618 
Item-13 678 Item-27 590 
Item-15 595 Item-28 570 
Item-16 419 Item-31 462 
Item-18 601 Item-33 516 


As can be seen in Table 3, as a result of the item-total correlation analysis, it was 
determined that the item factor loadings of the scale were higher than the accepted cut-off value 
of .30 (Biiyiik6zttirk, 2018). 

Cronbach’s alpha reliability test was used for the reliability analysis of the 5 sub- 
dimensions obtained as a result of the EFA. The results of the Cronbach’s alpha reliability 
analysis of the total scale and its sub-dimensions are given in Table 6. 


Table 6 
Internal Consistency (Reliability) Coefficients of the Total Scale and its Sub-dimensions 


Sub-Dimension (Factor) Cronbach’s Alpha 
1st Factor .920 
2nd Factor 817 
3rd Factor 857 
4th Factor 828 
5th Factor .610 
Total Scale .938 
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As can be seen in Table 6, the Cronbach’s alpha coefficient, which shows the internal 
consistency of the scale, is .938 for all items, while according to the sub-dimensions, it takes 
values between .920 and .610. According to the obtained findings, it was determined that the 
measurements obtained for the total scale were highly reliable (Ozdamar, 2004). 


Correlation Values between Factors 


The correlation values between the sub-dimensions of the problem-solving perception 
scale are given in Table 7. 


Table 7 
Correlation Coefficients Between Factors 


Factors Factor-1 Factor-2 Factor-3 Factor-4 Factor-5 
1st factor 1 .301** 422** 328** 349** 
2nd factor 301** 1 .647** 647** .094* 
3rd factor 422** .647** 1 612** .096* 
4th factor 328" .647** .612** 1 .084 
5th factor 349" .094* .096* 084 1 


As can be seen in Table 7, the correlation coefficient between the 5th factor and the 2nd 
factor is r= .094 (7 < .30), the correlation coefficient between the Sth factor and the 3rd factor is 
r= .096 (r < .30) and the correlation coefficient between the Sth factor and the 4th factor is r = 
.084 (7 < .30), while the correlation values between the other factors vary in the range of .30 <r 
<.80. The fact that the correlation coefficient is between .30 and .80 indicates the expected level 
of correlation between the sub-dimensions of the scale (Bitytik6ztiirk, 2018). On the other hand, 
since the correlation coefficient between the 5th factor and the 2nd, 3rd and 4th factors was not 
in the range of .30 <7 < .80, and by considering that the reason for the low correlation between 
the 5th sub-dimension and the other sub-dimensions might be that the scale items were not 
fully understood by the sample, it was decided to conduct a second pilot study. The expressions 
included in the 13th and 26th items, which were thought to be the reason for the low correlation 
of the sub-dimension named “Giving importance to problem-solving activities” with the other 
sub-dimensions, were corrected. In the second pilot implementation of the perception scale, the 
13th and 26th items were included as follows: 


Item 13 (previous form): I do not mind whether a large number of problem-solving 
activities are included in the mathematics lesson. 

Item 13 (latter form): I do not mind whether more or fewer problem-solving activities are 
included in the mathematics lesson. 

Item 26 (previous form): I do not mind whether there are problem-solving questions in 
the exams I will take. 

Item 26 (latter form): I do not mind whether there are a few or a lot of problem-solving 
questions in the exams I will take. 


The sample of the second pilot implementation, which was determined by convenience 
sampling, consisted of 252 secondary school students studying at a secondary school in Antalya. 
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As a result of the second pilot implementation, the same factor structure was confirmed, and it 
was again verified that the scale was valid and reliable. Correlation analysis was also repeated 
to determine the levels of correlation between the sub-dimensions of the scale. The findings 
obtained as a result of the analysis are given in Table 8. 


Table 8 
Correlation Coefficients between the Sub-dimensions of the Scale According to the Second 
Pilot Study 


Factors Factor-1 Factor-2 Factor-3 Factor-4 Factor-5 
1st factor 1 380" 495** 453** 636** 
2nd factor .380"* 1 .655** .658** 523" 
3rd factor 495** 655** 1 594** 549% 
4th factor 453** 658** .594** 1 568** 
5th factor .636** 523" .549** 568** 1 


As can be seen in Table 8, it was determined that the correlation values between 
the sub-dimensions of the scale ranged between .30 < r < .80. Accordingly, it can be said 
that the correlations between the sub-dimensions of the scale are significant and acceptable 
(Biytik6ztiirk, 2018). 


Discussion 


In this study, a scale was developed to determine the perception levels of secondary school 
students towards mathematical problem solving. In the literature, numerous measurement tools 
have been developed to determine students’ attitudes towards mathematical problem solving 
and the effects of these attitudes on their academic achievement (Aydin, 2020; Russo & Minas, 
2020). However, the great majority of these studies consists of achievement tests aimed at 
problem-solving (Fidan, 2008; Turhan, 2011). Students' attitudes towards problem-solving in 
different courses were also explored (Erdemir, 2009; Mason & Singh, 2016). In the Adesoji’s 
(2017) study, the effect of problem-solving strategies on students’ attitude toward chemistry 
was investigated. As a result of the study, it was suggested that teachers adopt problem solving 
strategies in their chemistry teaching. On the other hand, scales for determining students’ social 
problem-solving skills can also be found in the literature (Hawkins et al., 2009; Yaman & Dede, 
2008). Ekici and Balim (2013) conducted a scale development study to determine the general 
problem-solving perceptions of secondary school students. The scale they developed differs 
from the scale in this study in that it aims to determine individuals’ perceptions about solving 
the problems they encounter in their lives. 

The sub-dimensions of the “Mathematical Problem-Solving Perception Scale”: Fear and 
hesitation towards problem solving, showing interest in problem solving in class, interest in 
problem-solving activities outside of class, willingness for problem-solving activities in class, 
and giving importance to problem-solving activities. In the fear and hesitation dimension, 
which is the first sub-dimension of the perception scale for mathematical problem solving, the 
perception of fear and hesitation that students feel during problem-solving activities in lessons, 
and the levels of distress associated with these, are measured. Indeed, in studies in the literature, 
it is reported that the subject of mathematics creates fear and anxiety in students (Alkan, 2011; 
Keklikci & Yilmazer, 2013; Oztop & Toptas, 2017). Ufuktepe (2009) stated that factors related 
to the subject were effective in the emergence of fear of mathematics in students, and that 
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students’ thinking processes were interrupted due to the stress they experienced especially in 
solving mathematical problems. 

In the in-class interest dimension, which is the second sub-dimension of the scale, the 
students’ perception levels of being interested in problem-solving activities in class and feeling 
proud of themselves due to their success in the activities, are measured. In studies investigating 
students’ attitudes towards the subject of mathematics in the literature (Altuntas & Erigen, 2021; 
Uygun, 2010), the subject of mathematics and the problem-solving process were considered 
together, and it was stated that students’ perceptions of problem solving in mathematics lessons 
affected their interest in and enjoyment of the lessons. 

In the dimension of extracurricular interest, which is the third sub-dimension of the scale, 
students’ level of interest in studies outside the classroom, such as problem-solving homework 
and projects, is measured. Problem-solving skills are considered important in order to be 
successful in daily life as well as in mathematics class (Ozcan, 2016). For this reason, directing 
students’ problem-solving process towards extracurricular activities may be beneficial in terms 
of increasing students’ achievement in class and problem-solving success in life. Similarly, 
students’ perceptions of their extracurricular problem-solving responsibilities, as well as their 
perceptions of taking part in problem-solving activities in class, can play a decisive role in 
their success in the subject and in life. Studies revealing that students have positive attitudes 
towards solving mathematical problems (Ozgen et al., 2017; Taspinar, 2011) also report that 
students’ encounters with mathematical problems in their daily lives and their beliefs in the 
benefit and necessity of problem solving are effective. For this reason, in the willingness and 
giving importance sub-dimensions of the scale, students’ desire to participate willingly in in- 
class problem-solving activities and the level of importance they attach to problem-solving 
activities in the learning process are measured. 


Conclusions 


Determining students' perceptions of problem-solving is an important step to improve their 
mathematical problem-solving skills. For this reason, in this study, the “Mathematical Problem- 
Solving Perception Scale” was developed to determine secondary school students’ perceptions 
of mathematical problem solving. The trial form of the 33-item scale was implemented with 
the participation of 325 secondary school students studying in the determined population. 
Following EFA and reliability analysis, 9 items were removed from the scale. CFA was also 
performed for 24 items and 5 sub-dimensions. The correlation between the sub-dimensions 
was examined, but the correlation of one sub-dimension with the other sub-dimensions was 
found to be low. For this reason, a second application was made in a different sample consisting 
of 252 secondary school students. In line with the analyses made, it was calculated that the 
“Mathematical Problem-Solving Perception Scale” consisted of five factors, the explained 
variance was 66.15%, and the Cronbach alpha value for the whole scale was .93. Consequently, 
it was concluded that the scale would reveal the perception levels of secondary school students 
towards mathematical problem solving in a valid and reliable way. 

The study contains some limitations. Although the sample size is adequate, it was 
prepared only for secondary school students. Considering that mathematical problem solving is 
important for students studying at all levels, it can be recommended that similar scales suitable 
for different education levels be developed. Moreover, the study includes only quantitative data. 
It can be said that the findings obtained from this scale development study can be supported by 
studies based on the interview method, with the aim of determining students’ perceptions about 
problem solving. 

For the future studies, it can be recommended to develop the Mathematical Problem- 
Solving Perception Scale on different education levels. The scale can be used in descriptive 
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studies to reveal students' current situation regarding mathematical problem-solving perceptions. 
Moreover, it can be suggested that the scale be used as a data collection tool in experimental 
research to improve students' mathematical problem-solving perceptions. 
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Abstract 


Migrations and migrants have been in the focus of interest and activities of many governments, 
organizations, and institutions, both worldwide and in Croatia. According to international policies, 
children who seek or are under international protection have the right to education. To access the 
education system of the Republic of Croatia more easily, they are given the opportunity to attend Croatian 
language preparatory or remedial classes. 

The research aim was to determine the needs and challenges posed by the inclusion of a Syrian asylum 
seeker in a Croatian primary school. The qualitative approach was applied. The case study presents a 
ten-year-old Syrian girl, attending a primary school in Zagreb. The data were gathered with the use of 
semi-structured interviews with the girl's sister, teachers in the primary school, and her classmates. In 
order to gain a deeper insight into the girl's functioning in the school context, notes made by teachers 
during observation of the girls performance in the course of the educational process were gathered and 
analysed. Guidelines and recommendations were developed that could have a positive impact on quality 
inclusion of children refugees and asylum seekers in the Croatian education system. 

Keywords: children asylum seekers, educational process, qualitative research 


Introduction 


According to the global estimate by the International Organization for Migration (IOM), 
the UN agency for migration, the total number of child migrants amounts to 31 million, out 
of which approximately 13 million children are refugees, 936,000 are asylum seekers and 17 
million have been internally displaced (IOM, 2019). It has been estimated that out of the total 
number of international migrants in 2020, 13% were children under the age of 18 (Migration 
Data Portal, 2021). The term migration is usually associated with voluntary departure from 
the country, and in this case, return to the country is allowed. However, refugees and asylum 
seekers are forced to leave their countries. The 1951 Refugee Convention (The Geneva 
Conventions) is the key legal document that defines the term and rights of refugees (Bourgonje, 
2010). According to this document, refugees have the right to education, and they will receive 
fair treatment. 

Smooth integration of children who are asylum seekers into the education system 
poses a great challenge, both due to the lack of flexibility in the offered services and due to 
language and cultural barriers (European Commission, 2020). Karabacak (2020) mentioned 
that Syrian refugee students in Turkey come across behavioral, academic, socio-cultural, and 
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family barriers. Essomba (2017) pointed out that although the EU member states have laws 
which guarantee the right to education for children and young people from migrant families, 
these rights are jeopardized, and even violated. Most countries face the similar challenges and 
problems in the process of integration of migrant students into the education system, but there 
are differences between them (Stasitinaitiené et al., 2020). 

The Croatian legal framework, adjusted to international regulations, states that children 
who are seeking or have been granted the right to international protection (child migrants or 
asylum seekers) have the right to primary and secondary education, on equal terms with Croatian 
citizens (Law on International and Temporary Protection, Official Gazette 70/15, Article 58). 
The same article of this law states that a child will be given an opportunity to attend school 
within 30 days after filing a request, and to attend preparatory or remedial Croatian language 
classes (if a child does not speak Croatian well or does not speak it at all). Remedial classes 
in other subjects will be organized as well if there should be a need for that. The way in which 
child migrants and asylum seekers are integrated into the education system in the Republic of 
Croatia is regulated by the Ordinance on the manner of implementing the programme and tests 
of knowledge of asylum seekers, asylees, foreigners under temporary protection and foreigners 
under subsidiary protection, for the purpose of joining the education system of the Republic 
of Croatia (The Official Gazette, 89/08). For these children, educational institutions have an 
obligation to organize Croatian language learning classes lasting between three to six months. 
At the same time, a student can attend regular classes with students of the same age or those 
who attend roughly the same grade. Children who are seeking or have been granted international 
protection often attend classes with children who are younger, and not with their peers, because 
they enrol in school based on the results of verbal tests whose validity is, under the given 
circumstances, questionable (Buzinki¢, 2017). Croatian language curriculum for preparatory 
classes was designed for pupils who do not speak Croatian language well or who do not speak 
it at all. The planned number of lessons is 70 (The Official Gazette, 151/11). The aim of such 
curriculum is to help these pupils attend school, find their place in the school environment, and 
communicate well within and outside their community. The following skills are included in the 
curriculum: listening, speaking, reading, writing and mediation, which included participation 
in conversations, writing e-mails or text messages (Official Gazette, 151/11). It is questionable 
whether children can learn a new language adequately in 70 lessons and whether they can be 
relatively smoothly integrated into the educational process. It should be taken into account that 
a significant number of child migrants and asylum seekers have not used the Latin alphabet, but 
the Arabic alphabet, which is read and written from right to left. 

The European migrant crises broke out in 2015 and a large number of refugees have 
been arriving in Europe ever since. The Republic of Croatia does not have a long tradition of 
receiving migrants from third countries and is not a desirable destination for permanent stay 
for migrant families. In most cases, Croatia is just a stop on their way to Central European or 
Scandinavian countries. According to the data available from the Ministry of the Interior of the 
Republic of Croatia — MUP for 2021, the countries from which the largest number of migrants 
to Croatia originate are Afghanistan, Iraq, Iran, Turkey and Syria (MUP, 2021). 

In order to prevent the segregation of child migrants, it is necessary to promote 
intercultural openness, a positive multilingual policy, zero tolerance to discrimination, 
inclusion of culture-sensitive teaching content, active participation of parents, intercultural 
training for the teaching staff in which the quality of teachers is more significant than their 
number (European Commission, 2020). In most European countries, student migrants usually 
exhibit lower academic achievement and give a lower estimate of the benefits they have of the 
educational process than domestic students (Eurydice, 2019). According to the same source, 
there are statistically significant differences between primary school students who speak the 
language in which the teaching process is conducted in their home environment as well and 
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those who do not speak it at home. The students who do not speak that language at home have 
a weaker sense of belonging to school and state that they are exposed to bullying by their peers 
(ibid). An increase in the number of children refugees in the past few years makes this topic 
extremely important and current. Quality inclusion implies integration of a person into all areas 
of the society of the country to which a refugee has arrived, and in this paper, the focus area is 
education. 


Research Aim and Research Questions 


The aim of the research was to determine the needs and challenges posed by the inclusion 
process of a Syrian asylum seeker in a Croatian primary school. 
The research questions examined in this research were: 
1. What are the challenges posed by the inclusion of asylum seekers in a Croatian 
primary school? 
2. What are the suggestions for a more quality inclusion of asylum seekers in the 
Croatian education system? 


Research Methodology 
Case Study 


The case study was used in the research in order to present the level of inclusion of a 
girl named Lamija' in a Croatian primary school. Case study is dominated by naturalist, holistic 
and hermeneutic phenomenological interests (Halmi, 2005). An intensive study of an entity (an 
individual, a group, class, family, local community or even entire nation or culture) can provide 
numerous descriptions. In this process, a strong emphasis is placed on the subjective dimension 
and understanding, while entity is studied in real life situations. Data were collected in real life 
circumstances in order to show the complex environment in which the girl is growing up and in 
which she is receiving education. 

One of the case study types is observational case study which is frequently focused on 
a student, teacher or a smaller collective, and the main methods of collecting empirical data 
are various forms of observation and in-depth interviews (Halmi, 2005). The process in which 
interviews are conducted can be open or semi-structured, and its aim is to get the respondents 
express their attitudes, feelings, opinions, and reasons for their behaviour spontaneously (Halmi, 
2005). In order to achieve a meaningful interpretation of qualitative data, the researcher must 
possess knowledge and skills, and be disciplined and ready to invest effort. One of the challenges 
posed by qualitative analysis is finding a way for a creative synthesis of data (Patton, 2015). 
Qualitative analyses within the case study aim at establishing the causality using timeframe in 
order to recognize the phenomena which preceded a certain event or behaviour (Milas, 2005, 
p. 605). 


Participants 


In the current research, purposeful sample was used. Lamija, a Syrian asylum seeker, 
attended the third grade of primary school in Zagreb, the Republic of Croatia, in the school 
year 2019/2020. There were 24 students in the class, and Lamija was the only asylum seeking 
student. All other students were Croatian citizens. Lamija did not have a learning support 
assistant. 


1 In the paper, the names of the people are pseudonyms, in order to ensure their anonymity. 
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Professional and scientific papers were studied first, and then semi-structured interview 
forms were prepared. The data on the family background were provided by the girl’s sister, 
since their parents do not speak Croatian (or any other world language) at all. The girl’s sister 
is a secondary school student, and she understands and speaks Croatian fluently. She also 
translated to her parents the purpose of the research and its implementation. It was pointed out 
that anonymity would be granted and that they could change their mind and withdraw from 
the research at any time. Parents provided their consent to conduct the research and publish its 
results. 


Participant's Profile 


Lamija was born in Iraq, in the city of Anbar. Her family left the country due to war. 
The family has 7 members — parents and five children. Lamija is the second child in the family 
and is 10 years old. They came to Croatia in 2016, when Lamija was five. She enrolled in the 
kindergarten but did not attend it because her parents did not take her there. She says she cannot 
remember her childhood there much. With the help of volunteers, the girl started learning 
Croatian soon after their arrival. In the second grade, she started attending Croatian language 
classes for beginners - 70 lessons altogether. She attended the first and the second grade in 
another primary school. However, when the family moved house, she transferred to a new 
school, the one she is currently attending. In the first two grades, she attended after school 
care programme and stayed there after regular classes to study and do her homework. She is 
currently a third-grade student in a new primary school, where after school care programme is 
not organized for students in the third grade or older. The parents do not plan to stay in Croatia 
since they perceive it only as a transit country in which they plan to stay until they become 
eligible to enter Austria. 

In the Republic of Croatia, primary school is obligatory for all children who turn 6 by 
April 1 of the current year. It lasts 8 years and is organized into classroom teaching (Grades 1 — 
4) and subject teaching (Grades 5 — 8). In the first four grades, pupils are taught by one teacher 
- class teacher, who teaches Croatian Language, Maths, Nature and Society, Arts, Music, and 
Physical Education. However, the class teacher does not teach a foreign language and optional 
subjects, such as Religious Education. Students who attend Grades 5 — 8 are taught by subject 
teachers. 

Adults (school staff) included in the research were also selected on purpose. They were 
formally asked to participate in the research and were provided with its description. They were 
asked to give their contribution by participating in interviews and making observation notes. 
In that way, all the teachers who were teaching Lamija took part in the research. The class 
teacher, who conducts most of the teaching, spends most time in school with her (about 18 
hours per week). Both English Language teacher and Islamic Religious Education teacher have 
two lessons per week in the class which Lamija attends. In order to collect data on how Lamia 
performs at school from different sources, two students who spend time with Lamia during 
breaks and after school were also interviewed. 


Data Collection Tools 


In this qualitative research, the semi-structured interview method was applied, as well 
as participant observation. Triangulation of several data collection methods and multiple 
perspectives (those of Lamija, her sister, teachers, and classmates) obtained from the participants 
made it possible to create a more comprehensive picture (Yin, 2012) about inclusion of the girl, 
a Syrian asylum-seeker, in the Croatian school context. 
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Interview 


A semi-structured interview is an interview form composed of a series of questions 
designed to obtain specific responses from the respondents (Fraenkel et al., 2012). On the one 
hand, a semi-structured interview was used to set guidelines, and on the other hand to create 
enough space for the respondents to express their thoughts, feelings, wishes and attitudes. In this 
way, a part of the interview was in fact an informal discussion, which enables a deeper insight 
into situations that take place in a certain classroom on an everyday basis (Martone et al., 2018). 
The questions were designed after the relevant literature had been studied, in cooperation with 
two experts who had had experience with inclusion of child migrants in schools. 

The interviews with the girl’s sister and teachers lasted for about 30 minutes, while the 
interview with the girls who spend time with Lamija was shorter (lasting about 15 minutes). 
All interviews were carried out in a one-to-one form, after the classes, in the school premises 
and at the previously arranged time. During the interview with Lamija’s sister, data were 
collected about the family’s migration to Croatia, the functioning of the family in Croatia (with 
emphasis placed on the usage of Croatian language in the family context, and involvement of 
parents and other family members in school obligations), the problems the family had faced 
and support they had received. The interviews with teachers were focused on how Lamija 
functions in school (achievement of the learning goals and fulfilling school obligations, such as 
doing homework and communicating with adults and peers). The purpose of conversations with 
Lamija’s schoolmates was to collect data on her socializing with peers at school and outside 
school, their communication, common experiences, and feelings while hanging out. 


Participant Observation 


Apart from conducting the interview, all class teachers were involved in participant 
observation process and made field notes that they considered relevant for Lamija’s performance 
in school. The class teachers made notes during classes (but also during the breaks and before 
and after the classes) in order to track as closely as possible Lamija’s activity, her teaching 
content comprehension, interaction with other students and school staff, and verbal and 
nonverbal patterns of behaviour. The purpose of the notes was to gain a deeper insight into 
Lamija’s behaviour and to check to what extent the information obtained in the interviews 
matches the notes. 


Data Analysis 


The interviews were recorded and transcribed to textual notes. The data obtained in the 
interviews and the notes were analysed using the content analysis method. Patton (2015, p. 790) 
pointed out that “content analysis refers to any qualitative data reduction and sense-making 
effort that takes a volume of qualitative material and attempts to identify core consistencies and 
meanings’, and that “case studies, for example, can be content analysed”. 

While analysing the content of the interviews, each of the three researchers designed a 
coding scheme independently, and the codes were later compared in order to carry out analytical 
triangulation (Patton, 2015). By reducing, connecting, and putting into correlation the obtained 
data, the researchers created categories. Patton (2015) pointed out two criteria that should be 
taken into account in the process of category creation: internal and external heterogeneity. To 
meet the first criterion, the data within a certain category were connected, while in order to 
meet the second criterion, a clear distinction between the categories was made. Reliability was 
obtained through collaborative teamwork of all researchers. 
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Patton (2015) pointed out that case study enables the integration of all data obtained in 
the interviews into a coherent story about a person. In order to obtain the maximum reliability, 
some original statements of the respondents are presented in the paper. Furthermore, notes made 
by the teachers during participant observation process were analysed and coded according to 
the categories designed during the interview analysis. 

The data obtained in this study were organized in four challenging categories, using 
selective coding. Besides posing a challenge, these categories create an opportunity for the 
integration of child migrants in Croatian primary schools. The categories are: Communication, 
Teaching Process, Peer Relationships and Obstacles and Recommendations. 


Research Results 


The results are presented according to the obtained categories. The purpose of reporting 
on the categories mentioned above is to present the views of various participants in this process 
on the challenges of the integration process of asylum seekers in primary education. 


Communication 


One of the obstacles for inclusion of asylum seekers in Croatian primary schools is 
communication. It is extremely important, both for their integration into Croatian society in 
general and for their performance in school. As the teaching process is conducted in Croatian 
language, the understanding of spoken and written language is of utmost importance for 
children’s inclusion in classroom activities. Although the girl had finished two grades of primary 
school in Croatia, communication in Croatian language seemed to be a crucial problem. 

The class teacher pointed out: 


On the first day of school, the girl stood in front of the class, looking happy and with a smile on 
her face. We played a game in which students introduced themselves to her and she introduced 
herself to them. She was answering their questions using short sentences, which made it clear to 
me that she did not understand Croatian very well. It was necessary to design activities which she 
would understand and in which she would participate without having to use Croatian too much. 
Her father works at a car wash and does not understand Croatian much. He does not come to 
school. Her mother takes children to school and kindergarten in the morning, but she does not 
communicate with anyone because she does not speak Croatian, although she has been living 
in Croatia for four years. Children often translate conversations between the mother and us, 
teachers. However, the translation process may not be reliable, since children do not have a good 
command of Croatian either. In fact, in most cases, I am not sure if the child has interpreted all 
the important information correctly. 


Throughout the year, the class teacher noticed and recorded the following: 


She understands Croatian only partially, especially after holidays, because they speak only Arabic 
and watch TV programmes in Arabic in the family environment. 

It is difficult to find out from Lamija herself what it is that she does not understand. And she does 
not understand much more than the words which are explained in the textbook below a certain 
text... She often says herself that she knows what she would like to say, but she cannot remember 
the exact words in Croatian (for example, we were doing an exercise in which it was necessary to 
think of an adjective or verb that she would use with a noun ‘dance’). 


In addition to communication in Croatian language, there is also communication in 


English, during English Language classes. The English Language teacher (a subject teacher) 
noted: 
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The girl sometimes understands the instructions in English, but sometimes she does not. When 
I ask her if she understands, she almost always says she does, but she does not. When I check 
her work later, I see that she has not done everything. She rarely asks if she does not understand 
something. 

I think that for her, learning both Croatian and English is a matter of foreign language learning. 


Lamija attends Islamic Religious Education classes and the subject teacher pointed out: 


At the beginning of the school year, I covered a lot of the teaching content with the girl in Arabic. 
Islamic religion has a lot of Arabic prayers. I am familiar with the basics of Arabic, so with 
asylum seeking children (I teach several such children) I tried to establish communication in that 
language. At the end of each lesson, we teach them Croatian, while they teach us Arabic. 
[noticed that she speaks and understands a lot of Arabic, but unfortunately, she is not acquainted 
with the Arabic alphabet. Since I am, sometimes I spent some time during the class to work on 
that. However, her curiosity did not last long. 

The student was very self-confident in reciting the prayers and showing what she had learnt, 
but this self-confidence would disappear when she was supposed to acquire the knowledge in 
Croatian language. 


Lamia is aware of the difficulties she has in communicating in Croatian, but she still 
speaks it better than some members of her family: 


I know what I would like to say, but I cannot remember the exact Croatian words to say it. 
I often accompany my mother when she goes somewhere, because in that way, my mother feels 
safe, and I translate something to her if necessary. 


The Teaching Process 


The aim of the teaching process, as a systematic and organized activity involving both 
students and teachers, is to achieve the learning outcomes. These are the outcomes the students 
are supposed to achieve in each school subject and are prescribed at the national level. Teachers 
have autonomy in creating and adapting the teaching content to their students’ needs. What was 
observed in the analyses of all interviews was the achievement of the learning goals and attitude 
towards the activities. 

The class teacher describes Lamija’s participation in the teaching process during the 
school year, noticing the following: 


She does not speak Croatian and she participates in the classes with great difficulties during the 
first semester. She does her homework and studies at school because there is nobody who can 
help her with it at home, since her family members (mother, father, brothers and sisters) do not 
speak Croatian. She rarely does homework at home, although she marks the assigned tasks. An 
additional problem is the fact that she does not have a habit of reading at home. Sometimes, a 
volunteer from “Are You Syrious” association comes to her home and helps her catch up with the 
tasks she has failed to do, helps her learn the new content or do her homework. 


The class teacher tried to relate some parts of the teaching content to the child’s previous 
experience, but it did not yield success. The girl showed no progress, says the class teacher: 


... For example, during a Music lesson, I asked her to sing a chant in Arabic, but she could not do 
that. Her parents did not help her either. When we were analysing the symbols of the Republic of 
Croatia, I asked her to present the symbols of Iraq (the flag, coat of arms, anthem), but she was 
not prepared for the task, as she did not have any support at home. 
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Furthermore, the class teacher points out that the child is very successful in some subjects 
where artistic abilities and expression are very important. 


Lamija is very successful in Music and Visual Arts classes. She has a sense of rhythm and melody, 
and can express herself through visual arts beautifully, meeting all task criteria. 


In the first grade of primary school in Croatia, apart from their mother tongue, students 
start learning a foreign language. Most frequently, it is English. English Language teacher, 
speaking about Lamija’s attitude towards school, made the following note: 


From the very first lesson, it was obvious that Lamija is a quiet, extremely polite girl. However, 
I was hoping that she would open up and participate in the lessons more actively. Unfortunately, 
that has not happened. Lamija rarely volunteers to say anything. She will speak a few words 
only when I ask her specifically. Sometimes she looks for the correct answer in her classmate s 
notebook or waits for the class to read the answers and then she writes them down. She always has 
a smile on her face and never shows dissatisfaction, anger or that she is having a difficult time. 


Islamic Religious Education teacher noted: 


In the classroom, Lamija was mostly quiet, shy, and withdrawn. She would say something only 
when I asked her a question. I have to say that she had a very positive attitude to work, that 
she was hard-working and wanted to make progress. She did her tasks regularly and neatly and 
achieved the learning outcomes completely. 

I used to let her listen to religious songs in Arabic. She remembered the words and the tune and 
sang them happily. 


Speaking about the teaching and learning process, Lamija points out: 


Of all school subjects, I like Maths, Visual Arts and Music most. I like the coursebooks, especially 
Music coursebook. For me, Nature and Society is the most difficult subject, because I don t really 
understand it. I think it is more difficult for me to understand what I read on my own than what I 
hear and what the teacher tells and shows me. 

In the mosque, I am learning to read and write Arabic. I can write Arabic better than speak it. 
There are many differences between Croatian and Arabic in the shape of the letters and writing 
direction. I think my knowledge of Croatian is better than my knowledge of Arabic. I learn Arabic 
through Croatian — when I started learning Arabic, I used Croatian words, because I could not 
say or explain something in Arabic. 


Zana, Lamija’s classmate, who attends Islamic Religious Education classes with her, 
noticed the following: 


During Islamic Religious Education classes, Lamija seems to be more relaxed. Her brothers are 
there, too, and the teacher lets them answer in pairs and complete each other's sentences. 


Peer Relationships 
Socializing with peers is an important segment in the process of growing up. Children’s 
well-being is achieved, among other things, through quality relationships with peers. 


The class teacher, reflecting on the relationships between the girl and other students in the class, 
says: 
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Several girls in the class offered to sit with Lamija and help her in the first days at school. On the 
first day of school, they gave her a tour of the school and showed her the classrooms... It has been 
several months since Lamija came to our class, but she still never approaches a group of children 
on her own. I used to encourage her to join them or ask the girls to include her in the games. She 
built a closer relationship with 3 or 4 girls in the class, one of whom attends Islamic Religious 
Education classes with her. However, she always keeps to herself and does not join them unless 
they invite her. She does not hang out with children from the class outside school, does not come 
to their birthday parties, does not visit anyone, and does not invite anyone to her home. 


English Language teacher: 


There are several girls in the class who are happy to help her when they finish their work. Lamija 
accepts their help gladly. 


Islamic Religious Education teacher: 


Lamija shows a great wish and is thrilled to teach her friends in Islamic Religious Education 
classes how to say something in Arabic. She converses happily with her classmate, they often chat, 
play, and laugh. 


Lamija: 


In my first school, I became friends with a girl named Ana. One weekend this winter, I went to her 
house with my younger sister and stayed there till the evening. We had a great time!... I rarely go 
out to play because my mother and father are worried that something might happen to me. I never 
go out with friends, but only with my younger brothers, for a short time... I like Zagreb and life in 
Croatia very much and I do not want to leave, but my parents are talking about going to Austria. 


Lamija’s classmates, the girls who spend more time with her than other children, notices 
similar patterns of behaviour in the school context, but outside it as well. 
Zana: 


At the very beginning of the school year, I noticed that Lamija has a beautiful smile, but that she is 
much more shy than other girls. I thought it was because she was a new student in our class, but 
Lamija has not changed. During the breaks, Katarina, Lamija and I spend time together. When I 
come to school, I approach Lamija and we start talking. I have never played with her outside, in 
our free time, because Lamija does not come outside to play. 


Katarina: 


I wanted to help Lamija so she would not feel alone and to help her get to know other students 
from the class more easily. At the beginning, Lamija was very quiet and shy, and she always waited 
for us to invite her to play with us. She used to accept whatever other girls suggested, but she 
would never suggest playing any game herself. 

I invited Lamija to come to my home several times, or to come outside and play. However, she 
never came. I did not ask her why, but she told me that she is allowed to go out only sometimes, 
with her brother and sister, and that she can stay outside only for a very short time. 


Obstacles and Recommendations 
During the analysis of the personal accounts and notes, it was noticed that some participants 
express concerns about the quality of the inclusion process. Obstacles and shortcomings were 


noticed both at the micro and the macro levels. The mere fact that they were noticed can be 
taken as an indicator pointing toward possible future activities and actions. 
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The class teacher’s observations point to the difficulties and identify support options necessary 
in working with refugee and asylum seeking children: 


Children in the class were able to explain who asylum seekers are; they have heard about the 
wars in Iraq and Syria, as well as about the asylum seekers who have travelled for a long time 
having fled their countries, sailing the seas in boats, and losing their lives... I was surprised to 
learn how much some of them knew. When I asked them where they had heard all those things, 
they said they had seen it all on TV. Luckily, Lamija did not have such a terrible experience 
leaving her country. However, before she came to our school, I knew nothing about her journey to 
Croatia. I spent a lot of time thinking about what and how to ask her about such things, without 
triggering some bad memories or trauma... Luckily, Lamija did not experience anything of the 
kind. The greatest support was exchange of experiences with colleagues from other schools, who 


have already taught asylum seeking children. On the other hand, support from the Ministry of 


Education and the Teacher Training Agency is insufficient, and there are no appropriate training 
sessions for teachers who work with such children. 


Islamic Religious Education teacher: 


I know that other colleagues mentioned communication as the biggest obstacle for the full 
adjustment and quality work with asylum seeking children, but I have not felt it that way teaching 
my subject. I ascribe it to the fact that Islam is something close and familiar to them. A great part 
of the teaching content is in Arabic. 

Reflecting on the entire school year, I would say that the biggest problem asylum seeking children 
are facing is the sense of not belonging anywhere. They are foreigners in their own country, 
among their own people and culture (because they do not speak the language, they do not know 
the alphabet, and are unfamiliar with the customs and religion), and they feel even more like 
foreigners in a new environment, the customs of which they are yet to learn and adjust to. 


For Lamia, the most difficult moment was when the school transferred to online teaching, 
that is, when distance learning started: 


The worst period for me was distance learning. Although I was not afraid of anything while being 
at home, it was still difficult for me, and I missed school and my friends. The teacher was not there 
to help me when I did not understand something or when I did not know how to do something. My 
sister could not help me either, because she had a lot of her own work to do. 


Discussion 


This study focuses on needs and challenges posed by the inclusion process of a Syrian 
girl, an asylum seeker in a Croatian primary school. Based on the findings of the present study, 
four categories were created: Communication, Teaching Process, Peer Relationships, and 
Obstacles and Recommendations. 

The results indicate that, although the girl had attended Croatian language preparatory 
classes (70 hours), it was not sufficient for her successful involvement and activity in class 
work. The girl herself points out that sometimes she finds it difficult to find the right words to 
express her opinion. Language is probably the most significant factor that enables a student to 
understand what is going in the classroom and to assess whether the content is relevant to him/ 
her or not (Chen & Gay, 2020). Successful mastery of a language, both its written and spoken 
form, is a precondition for successful inclusion in society and functioning in everyday situations. 
Communication is the basis of inclusion of students in the class, and inclusion of family in the 
neighbourhood and local community. Aydin and Kaya (2017, p. 13) thought that teaching the 
language spoken in a country is the priority in educating migrants and refugees around the 
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world. Inadequate knowledge of Croatian language makes communication and participation in 
the teaching process difficult, which results in reduced activity in the class, partial achievement 
of the learning goals and lack of motivation. Aydin and Kaya (2017) analysed educational needs 
and barriers encountered by Syrian refugee students in schools in Turkey. They pointed out that 
integration depends on their level of knowledge of Turkish language and that it takes 3-4 years 
to achieve verbal fluency, while it might take 4-7 years to achieve academic fluency. Other 
researchers have obtained similar results, pointing out that in the optimum circumstances, it 
takes between three and five years to develop oral language proficiency, while it takes between 
four and seven years to achieve academic proficiency in English language (Collier, 1989, 1995; 
Hakuta et al. 2000; according to Miller, 2009). Teaching migrant students, the language in which 
the teaching process is conducted poses certain challenges because this language is frequently 
the second or additional language they need to master at a high level, in order to be able to learn 
other subjects (Eurydice, 2019, p. 11). In order to facilitate second language acquisition, it is 
necessary to design appropriate programmes, to have support of the relevant institutions and 
to maintain partnerships between all participants in the educational process (Pavlicevic-Frani¢, 
2011). Better linguistic preparation of migrant students would facilitate the work of teachers, 
who also face difficulties in communication. Some of the concrete suggestions for language 
learning and education of migrant children were based on the results of research carried 
out in Croatia within the project “Supporting the integration of third-country nationals who 
need international protection”. These suggestions are: organize adequate Croatian language 
preparatory classes in schools; prepare a starter coursebook for Croatian language learning 
and translate it into three languages most frequently used and understood by asylum seekers 
(Arabic, Farsi and Kurdish); employ more teaching assistants to help asylum seeking children 
in the teaching and learning process; allow for more hours for learning Croatian language in 
schools, and provide financial support for the specific school needs of asylum seeking children 
(Ajdukovié et al., 2019, p. 97). 

The main findings about the teaching process reveal that the girl manages to achieve the 
learning outcomes, but at different levels. She is not equally successful in all school subjects. 
Quality cooperation between the school and the girls’ parents has not been achieved, because 
the girl’s mother avoids school, does not take part in any activities related to school, does not 
help her child with homework and it is obvious that the family does not provide support for 
the girl with school tasks. The girl is able to acquire more quickly those parts of the teaching 
content which correlate to her previous experience, and which are associated with positive 
events in her life. In these cases, she is more active and motivated to work. A significant 
number of immigrants wish to maintain their social identities and cultural engagements, 
which include language (Baysu et al., 2011). However, in order to participate in the teaching 
process successfully, it is necessary to acquire the language of the country they are currently 
living in. Weiss et al. (2018) emphasize concerns expressed by teachers and schools regarding 
academic achievements of migrant children, their behaviour in society and a possibility of 
quality interaction between schools and parents. Migrant children frequently have difficulties 
with cultural, social, and academic requirements in school, and sometimes even with their own 
unrealistic expectations (Miller et al., 2005). Students in compulsory education need education 
full of strong emotions and a strong affective component; however, schools and teachers are not 
prepared for offering this kind of support (Essomba, 2017). 

As discovered in this study, teachers estimated the girl’s participation in the teaching 
process differently. They generously shared their observations with us, hoping they would shed 
light on some aspects of inclusion of migrant and asylum seeking children in our schools. 
This would be a step towards a quality inclusion process. It should be pointed out that the 
girl functioned better in Islamic Religious Education classes, probably because the subject is 
culturally and linguistically closer to her. Another reason might be the fact that the Islamic 
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Religious Education teacher is in contact with a greater number of refugee and asylum seeking 
children, as most of them attend these classes. An encouraging and supportive relationship 
between teachers and migrant children might have a key role in these children’s adjustment to 
the school context and have multiple benefits for them (Hoti et al., 2015). Apart from that, Hoti 
et al. (2019) presented the results of the research they had carried out with 1,106 students and 
their teachers in Switzerland. They pointed out that students are more satisfied with school if 
they share the common values and attitudes towards cultural diversity. The specific problem was 
online teaching, organized for the last three months of the school year, due to the COVID-19 
pandemic. For most teachers, as well as students, online education during the pandemic posed 
a great challenge (Osmanovié Zaji¢ et al., 2022). During that period, the girl lost live contact 
with the teacher and students, and the support she received in the family environment was 
inadequate. 

The results do not draw attention only to the academic needs of the girl, but also to her 
emotional and social needs (socializing with peers). Unfortunately, although the girl is in the 
third grade, she has already changed school because her family moved to another flat. Therefore, 
the continuity in her education was disrupted, while going to another school, in already unstable 
conditions in which her family has been living (gaining a certain status and moving to Austria), 
made her socialization process more difficult. The school year 2019/2020, in which the girl’s 
work was observed, was her first year in the new school. It seems that she did not manage to 
make any close friendships with other students. Other students took the initiative, and their 
socializing was pleasant. However, socializing takes place only in the time before coming to 
school and during the breaks. In her free time, the girl does not spend time with other children 
due to her parents’ rules, although she would like to. Makarova and Birman (2016) stressed the 
importance of measures and strategies which promote interaction between migrant children and 
native speakers, and which enable successful adaptation. 

The results of this study pointed to certain difficulties and a need for systematic and 
coordinated support for all participants in the educational process: migrant children, teachers, 
school staff and parents. Unfortunately, national and school curricula do not include programmes 
for acquiring linguistic skills and social competencies that are necessary for successful 
integration of migrant children into society. During their initial education, the teachers are not 
acquainted with the ways to provide support for such children. Teachers and school staff are not 
given systematic support and an opportunity to develop the necessary competencies to accept 
and include these children into school. Teacher education programmes do not train teachers in 
line with the key concepts of multicultural education (Gorski, 2009), and it is not surprising that 
teachers feel unprepared for and insecure in work with such children (Sinkkonen & Kyttala, 
2014). Pugh et al. (2012) stressed the importance of providing the teachers with guidance and 
support through various continuing professional development programmes, in order to make 
them competent to overcome diverse obstacles they can come across while teaching migrant 
children. School as an institution cannot meet the needs of migrant and asylum seeking children 
on its own. Pastoor (2017) believed that participation of migrant children in various types of 
environment and activities outside the school context creates more opportunities for meaningful 
learning, promotes their social inclusion, improves the educational process, and facilitates 
psychological and social adjustment. According to the report by European Commission (2020), 
Croatia has a solid policy framework for promoting children’s rights and well-being. However, 
it lacks good coordination of various bodies and agencies at all levels, in order to achieve the 
set goals. 
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Conclusions and Limitations 


The findings of this study demonstrate that insufficient knowledge of Croatian language 
is a great obstacle in achieving quality communication, both with adults and peers. It makes 
active participation in the teaching process difficult and prevents successful achievement of the 
learning goals. Croatian Preparatory Language Learning and Latin script learning programmes 
for child refugees and asylum seekers should be intense and comprehensive in order to enable 
students not only to communicate when necessary, but also to enable them to participate easily 
in the learning and teaching process. The current 70-hour Croatian Preparatory Language 
Learning Programme is inadequate and does not enable students to overcome the language 
barrier. It is necessary to design quality Croatian language learning courses, but also to 
train teachers for work with migrant children and asylum seekers. Teachers need systematic 
support to help them ensure smooth inclusion of such children in the classroom. Furthermore, 
cooperation with parents should be given more attention in order to facilitate adjustment of 
children to new circumstances. Local communities should also become involved, in order 
to make sure the migrant family is not left on its own, isolated and confined within its own 
four walls. Civil society organizations could be helpful in ensuring quality inclusion of child 
refugees and asylum seekers, not only in the education system of the host country, but also in 
other spheres of social life. 

Migrant crises, no matter in which part of the world they start, and regardless of the 
cause, result in displacement of children, who then need to be included in education systems 
of the countries in which they find themselves. Frequently, these are the countries in which 
a different language is spoken, a different type of alphabet is used, and different secular and 
religious laws are obeyed. Croatian education system has not so far experienced the inclusion 
of third-country refugee children and it has no previous experience in creating the necessary 
programmes. Refugee and asylum seeking children are vulnerable groups and need to be given 
an opportunity to meet all their specific needs and develop their full potential. It is necessary 
to organize a more comprehensive exchange of experiences between experts in various fields 
(teachers, pedagogues, psychologists, linguists, social and health care workers, etc.) in the 
country, but also create opportunities for networking at the international level, in order to 
facilitate quality inclusion of children. This case study is unique in the Croatian context, but 
its results can be applied in other countries which receive child refugees and asylum seekers. It 
also enables a better understanding of some challenges posed by inclusion of child refugees and 
asylum seekers in schools in host countries. Future research should focus on identifying other 
indicators of successful inclusion. 

Since this was a type of research in which only one participant was observed, we are aware 
that it is not possible to make any generalizations. However, it does present a way of collecting 
valuable data illustrating the inclusion of asylum seeking children in compulsory primary 
school. In order to minimize potential faults in the application of case study, triangulation of 
the methodological approaches was applied. The methods used were participant observation 
and interviews. Furthermore, we are aware that the availability of the collected data is limited 
because the girl finds it difficult to express her thoughts and feelings in Croatian/English 
language. That is why the inclusion of teachers who speak the same language as the girl or 
teachers who have competencies for teaching Croatian language as a foreign language might 
facilitate understanding. 
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Abstract 


With the start of the COVID-19 pandemic period, the need to continue the educational process imposed 
the exclusive move to the online environment - a new and insufficiently prepared experience, which 
generated controversies, but also reflections, and adaptation for teachers, students, decision-makers, and 
even parents. The major concern of the academic community was related to the quality of the educational 
process and how the student-centered education paradigm can be introduced in the online environment. 
The students and teachers of the post-pandemic generation are no longer the same, this period has 
strongly impacted society, mentalities, and the school's future. Analyzing online education from multiple 
perspectives, the advantages, and opportunities generated by the integration of ICT in the teaching 
process are obvious. However, many variables have an impact on the effectiveness of e-learning, and 
many unanswered questions, yet. This research aims to analyze the key drivers of technology acceptance 
and the role played by three external variables: content adaptation, facilitating conditions, and ease of 
access in the context of exclusive online education. For this purpose, an extended technology acceptance 
model has been conceptualized and tested on a sample of Romanian university students. The results 
illustrate that content adaptation represents an important predictor of both the perceived ease of use and 
usefulness. The model explained a lot of variance in the continuance intention which is due to the positive 
attitude towards using the online learning platform in the future. 

Keywords: distance education, TAM, COVID-19 pandemic, online learning platform, learning motivation 


Introduction 


With the new reality imposed by the COVID-19 pandemic, online education has become 
a viable alternative and an open door to the future, in which school will no longer be just face- 
to-face, in a traditional format, but will effectively be capitalized by educational platforms, 
resources, and means of modern technology, to produce creative and innovative learning, to 
stimulate and maintain students’ motivation, interest and desire for knowledge. 

Although the implementation of the online teaching-learning process has been achieved 
with effort, risks and difficulties, in many cases it has also brought satisfaction and success, 
constituting a bridge over the physical distance between teacher and students and mediating 
learning. Analyzing several studies from the pre-pandemic period, Nguyen (2015) stated that, 
if properly designed, online courses can be as effective as traditional ones. 
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But the pandemic context, with implicitly imposed strong emotional load and restrictive 
social measures, with isolation in many situations, unpredictability, and full of anxiety, 
generated negative effects on education - stress, low interest, lack of motivation, delays, low 
productivity, and reduced engagement in the online learning process. In this respect, not all 
students responded positively to the implementation of online learning (Duraku, 2021). 

During the COVID-19 pandemic, students’ behavior and attitude toward online learning 
are influenced by many factors, as mentioned by several researchers (Boca, 2021; Um, 2021). 
However various studies have shown that students have gained knowledge through online 
education during this period, an appropriate pedagogy and suitable content are important 
variables of students’ learning motivation (Boca, 2021; Hamdan & Amorri, 2022). 

A wide range of course and learning management systems (CMS / LMS) are used in 
education - platforms that offer synchronous or asynchronous support and include advanced 
features for facilitating student access to resources and learning materials, mediating 
collaboration and communication, evaluating and providing feedback, and many other 
facilities: Moodle, Google Classroom, Edmodo LMS, CANVAS LMS, Schoology, Sakai, 
EdApp, Thinkific, ZoomLearn, Teachable (https://research.com/software/best-free-learning- 
management-systems). 

Anyway, in the context of the pandemic, the shift to exclusive online education represented 
a challenge not only for educators but also for researchers interested in explaining the adoption 
and usage of e-learning systems. In this sense, the technology acceptance model (TAM) 
developed by Davis (1989) stated that the intention to use a given technology is determined by 
two users’ beliefs: the perceived ease of use and the perceived usefulness. The influence of the 
two beliefs is mediated by attitude, which has been defined as affective or evaluative feelings 
about behavior (Azjen, 1991). 

This research aimed to analyze the determinants of the acceptance of an online learning 
platform by university students and the role played by three external variables: content 
adaptation, facilitating conditions, and ease of access. For this purpose, an extended TAM- 
based model has been conceptualized and tested on a sample of Romanian university students 
(N = 156). 


Background and Conceptualization 
Background 


TAM considered that the behavioral intention to use a computer system is driven by the 
attitude towards usage which, in turn, is influenced by two beliefs: perceived ease of use (PE) 
and perceived usefulness (PU). Extant literature shows a large variety of technology acceptance 
models as well as a large diversity of conceptualizations as regards the main drivers and their 
antecedents (Lee et al., 2003; Granik & Marangunic, 2019). 

As an example, the study of Pal and Patra (2020) integrated the technology acceptance 
model with the task-technology fit model (TTF) by conceptualizing TTF as an antecedent of 
the perceived ease of use and perceived usefulness of online video-based learning. The task- 
technology fit has been conceptualized as a latent variable with two antecedents: technology 
characteristics and individual characteristics, the last one having a larger influence on TTF. The 
influence of TTF on the perceived ease of use was higher than the influence on the perceived 
usefulness. 

The task-technology fit has been also included in a TAM model by Mo et al. (2021). 
They tested an extended TAM model including three external variables (task-technology 
fit, instructor attitude, and family support) on a sample of 552 university students. The task 
technology fit was the only significant antecedent of the perceived usefulness. All external 
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variables were significant antecedents of the perceived ease of use. The model explained a 
55.9% variance in the continuance intention. 

Baber (2021) analyzed the intention to use an online teaching and learning system (Zoom 
+ LMS) during the pandemic. He developed and tested a hierarchical model having three 
second-order formative constructs that are measuring instructor features (attitude, competency, 
interaction), student features (motivation, collaboration, and student mindset), and technology 
acceptance (perceived ease of use and perceived usefulness). The intention to use was mainly 
influenced by the instructor's characteristics and the student's characteristics. The influence of 
technology acceptance was smaller. Overall, his model explained 76% variance in the intention 
to use. 

Mailizar et al. (2021) examined the behavioral intention to use e-learning in a university 
in Indonesia. They conceptualized an extended TAM model featuring two external variables: 
system quality and e-learning experience. Their results showed that attitude was the only 
predictor having a crucial influence on the behavioral intention to use. As regards the external 
variables, only system quality had a significant influence on the perceived ease of use and 
perceived usefulness. The model explained a 57% variance in the intention to use. 

Akuratiya and Meddage (2020) analyzed the IT students’ perceptions of online learning 
in Sri Lanka. They found a high perception of online learning (79%) and a high agreement 
to integrate online learning in the future. Most of the students found online learning useful 
(77%), effective (67%), comfortable (53%), and enjoyable (51%). Baczek et al. (2021) reported 
similar perceptions of medical students in Poland: comfortable (54%), learning at their own 
pace (64%), access to online material (68%), and enjoyable (73%). 

Analyzing the students’ acceptance of e-learning in academic studies, by considering 
a modified TAM model that takes into account the instructor characteristics, computer 
self-efficacy, course design, perceived usefulness, perceived ease of use, and intention to use, 
95 undergraduate students from Malaysia expressed that computer self-efficacy, perceived ease 
of use, and intention to choose the e-learning format for study, represent crucial factors that 
affect the students’ use of e-learning in higher education (Ibrahim et al., 2017). 

Anyway, the development of comprehensive TAMs allows research on e-learning 
acceptance. A positive example is coming from UAE, where 435 students from 5 universities 
participated in research that illustrated a significant impact on the perceived ease of use of 
the e-learning system, influenced by the quality of the system, computer self-efficacy, and 
computer playfulness. Near the abovementioned variables, the quality of information, perceived 
enjoyment, and accessibility proved to induce a favorable influence on the perceived ease of use 
and perceived usefulness of the e-learning system (Salloum et al., 2019). 


Research Model and Hypotheses 


In this study, an extended TAM is proposed that includes two main beliefs influencing 
the behavioral intention to use (BI) through the attitude toward use (AT): perceived ease of use 
(PEU) and perceived usefulness (PU). It has been hypothesized that the perceived ease of use 
and perceived usefulness are influenced by two external variables: content adaptation (CA) and 
the ease of access to online activities (EA). The research model is presented in Figure 1. 
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Figure 1 
Research Model 
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The ease of access refers to the new opportunities created by online education: the ubiq- 
uitous and fast access to the educational activities and educational resources stored on the 
platform (Lamanauskas & Makaraskaite-Petkevitciene, 2021; Manea et al., 2021). Therefore, it 
has been hypothesized that the ease of access is positively influencing the perceived ease of use. 


H1 Ease of access has a positive influence on the perceived ease of use (EA— PEU) 


Facilitating conditions refer to the availability of resources, such as computers, Internet 
access, and the knowledge needed to use the online platform. It is expected that facilitating 
conditions will positively influence the perceived ease of use and the perceived usefulness 
(Almayah et al., 2020; Clark et al., 2020) 


F122 Facilitating conditions have a positive influence on the perceived ease of use (FC— PEU) 
#3 Facilitating conditions have a positive influence on the perceived usefulness (FC— PU) 


Content adaptation refers to the suitability of the content for online presentation and the 
suitability of assignments for online learning. On the one hand, suitable content has a positive 
influence on the perceived ease of use since it minimizes the information density on the screen, 
makes it easier to read and/or follow, and reduces cognitive load. On another hand, suitable 
content favors an easier and better understanding of the learning content thus having a positive 
influence on the perceived usefulness while a lack of content adaptation makes it difficult for 
students to keep their attention and concentrate on the learning task (Lamanauskas & Makaras- 
kaite-Petkevitciene, 2021). 


H4 Content adaptation has a positive influence on the perceived ease of use (CA— PEU) 
HS Content adaptation has a positive influence on the perceived usefulness (CA— PU) 


Davis (1989) defined the perceived ease of use (PEU) as “the degree to which a person 
believes that using a particular system would be free of effort” (Davis, 1989). It is expected 
that students will find an easy-to-use platform more useful for learning (Lee et al., 2005; Pal & 
Patra, 2020). It is also expected that the perceived ease of use will make learning more pleasant 
for students (Lee et al., 2005) and will make them more interested to use the learning platform 
(Yi & Hwang, 2003). 


H6 Perceived ease of use has a positive influence on the perceived usefulness (PEU— PU) 
H7 Perceived ease of use has a positive influence on the attitude toward use (PEU— AT) 
H8 Perceived ease of use has a positive influence on the intention to use (PEU— BI) 
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Perceived usefulness has been defined by Davis as a belief that using a given technology 
will improve job performance (Davis, 1989). It is supposed that the perceived usefulness will 
positively influence the attitude toward using the system and the intention to use it (Lee et 
al., 2005; Lazar et al., 2020; Raza et. al, 2020; Baber, 2021). In turn, a positive attitude will 
influence the intention to use (Azjen, 1991; Davis, 1989). 


H9 Perceived usefulness has a positive influence on attitude (PU — AT) 
[10 Perceived usefulness has a positive influence on the intention to use (PU — BI) 
HII Attitude has a positive influence on the intention to use (AT — BI) 


Research Methodology 
General Background 


The students enrolled in the university study programs benefited from the support of Moodle 
as a course management system (CMS). In this respect, the platform provided management and 
contact with all students and offered documentation for each discipline and communication 
facility at the individual/group level. In addition, the platform offered possibilities for online 
evaluation and is also interfaced with the Microsoft Teams application, which allows student- 
teacher interaction, online development of courses, seminars, and application classes, as well as 
organization of face-to-face evaluation (Bizoi et al., 2017; Santi et al., 2020). 


Instrument and Procedure 


The instrument designed for evaluation has been developed considering previous 
research related to online learning during the pandemic (Lamanauskas et al., 2019; Pribeanu et 
al., 2020), and also by adapting existing scales in the TAM literature (Azjen, 1991; Davis, 1989; 
Davis et al., 1992; Lee et al., 2005). 

The model was tested with Lisrel 9.3 for Windows (Mels, 2006), using the maximum 
likelihood estimation method. Before model testing, an exploratory factor analysis (maximum 
likelihood with varimax rotation) has been carried out that revealed several items with poor 
factor loading. Therefore, these items have been eliminated from the scale. Table 1 presents the 
constructs and the remaining items. 
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Table 1 
Measures 
EA Ease of access 
EA | can access the online learning platform anytime from anywhere 
EA2 Logging on to the online platform is easy 
FC Facilitating conditions 
FC1 | have the resources to use the online learning platform 
FC2 | have the knowledge to use the online learning platform 
CA Content adaptation 
CA The content is adapted to online presentation 
CA2 Students’ assignments received are adapted for the online learning platform 
PEU Perceived ease of use 


PEU1 Using the online learning platform is simple 


PEU2 Learning how to use the online learning platform is easy 


PEU3 My interaction with the online platform is clear and understandable 


PU Perceived usefulness 

UU1 Using the online platform will improve my learning performance 

UU2 Using the online platform makes it easier for me to study 

UU3 Using the online platform makes learning more productive 

AT Attitude 

AT1 | believe that using the online learning platform is interesting 

AT2 | believe that using the online learning platform is pleasant 

Bl Behavioral intention to use 

Blt | intend to use the online learning platform in the future 

Bl2 | will use the online learning platform if it is available after the pandemic 


Convergent validity has been assessed according to the recommended thresholds from 
the literature (Fornell, & Larcker, 1981; Hair et al., 2010): loadings magnitude greater than 
0.5, composite reliability (CR) greater than 0.70, and average variance extracted (AVE) great- 
er than 0.5. Discriminant validity has been assessed through the squared correlation test (For- 
nell & Larcker, 1981). 

The model fit with the data has been assessed by using the following goodness of fit in- 
dices (Hair et al., 2006): chi-square (x), degrees of freedom (df), y’/df, comparative fit index 
(CFI), the goodness of fit index (GFI), root mean square error of approximation (RMSEA), 
and standardized root mean square residual (SRMR). 


Sample 


The data has been collected during the university year 2021-2022. The instrument 
designed for evaluation purposes has been applied to third-year students enrolled in the 
psycho-pedagogical study module (initial training for future teachers) at Valahia University in 
Targoviste, Romania. 

A total of 161 undergraduate students answered the questionnaire. The data has been 
analyzed for incomplete data and 5 cases have been eliminated thus resulting in a working 
sample of 156 cases (46 men / 110 women). As regards the background, students were coming 
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from several faculties, covering on a large scale the palette of Valahia University specializations: 
education (50), physical education and kineto-therapy (27), music (20), sciences (17), 
engineering (12), economics (11), theology (10), and other (9). 


Research Results 
Model Testing Results 


The model has been analyzed for unidimensionality, the internal consistency of the scale 
(Cronbach’s alpha), and convergent validity. The descriptive statistics and the standardized 
factor loadings are presented in Table 2. The mean values for the ease of access and perceived 
ease of use are high which suggests that university students are familiar with the online learning 
platform. All item loadings are over 0.70, thus proving unidimensionality. 


Table 2 
Descriptive and Item Loadings (N=156) 


Factor Item M SD Loading 
EA 4.38 1.07 Wis) 
EA 
EA2 4.38 0.99 96 
FC1 4.40 1.02 87 
FC 
FC2 4.32 1.04 98 
CA1 3.97 1.08 84 
CA 
CA2 3.97 1.17 13 
PEU1 4.27 1.03 94 
PEU PEU2 4.22 1.07 93 
PEU3 417 1.06 93 
PU1 3.64 1.21 .90 
PU PU2 3.65 1.30 93 
PU3 3.49 1.30 92 
AT1 3.92 1.12 93 
AT 
AT2 3.76 1.29 92 
BI 3.60 1.42 93 
BI 
Bl2 3.49 1.52 91 


The scale reliability, convergent, and discriminant validity are presented in Table 3. The 
Cronbach’s alpha is varying from .756 to .953, thus showing good reliability of the scales. 
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Table 3 
Scale Reliability, Convergent, and Discriminant Validity (N=156) 


Alpha = CR AVE CA EA FC PEU —- PU AT BI 
CA 0.756 0.757 0.609 ~—0.782 
EA 0.836 0.850 0.742 0.346 0.861 
FC 0.918 0.953 0859 0.514 0.623 0.927 


PEU 0.952 0.951 0.865 0.483 0.845 0.730 0.933 


PU 0.939 0.938 0.834 0600 0335 0477 0.437 0.913 
AT 0.943 0.949 0903 0455 0.336 0415 0456 0.735 0.950 
Bl 0.916 0.917 0.847 0.351 0.225 0.377 0.360 0.768 0.955 0.934 


Note: The bold diagonal numbers represent the square root of AVE 


The composite reliability (CR) and average variance extracted (AVE) are over the cut-off 
values of 0.7, respectively 0.5 (Fornell & Larcker, 1981) thus proving the convergent validity. 
The discriminant validity is acceptable, since (with two small exceptions) the correlation 
between factors is smaller than the square root of AVE. 

The model fit with the data is excellent, as shown by the goodness of fit (GOF) indices: 
nonsignificant ¥?, df=90, y?/df=1.165, CFI=0.994, GFI=0.922, RMSEA=0.032, SRMR=0.0293. 
The model estimation results are presented in Figure 2. 


Figure 2 
Model Estimation Results (N = 156) 


an 


Chi-Squeres104.89. df*90. P-value*0 13502. RMSEA+0.033 
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The paths from FC to PU, from PEU to PU, and from PEU to BI are not significant so 
the hypotheses H3, H6, and H9 are not supported. This suggests that the ease of access and 
ease of use are taken for granted in an online learning context and therefore not relevant for the 
perceived usefulness. In the same way, it is explained the nonsignificant influence of PEU on 
the intention to use. 

The hypotheses H1, H2, H4, H5, and H7 are supported since the paths from EA to PEU 
(B = 0.63, p < .001), FC to PEU (B = 0.27, p < .001), CA to PEU (B = 0.50, p < .001), CA to PU 
(B=0.46, p<0.001), and PEU to AT (B = 0.16, p = .018) are significant. This shows that content 
adaptation is a relevant variable for both perceived ease of use and perceived usefulness. 

PU has a significant positive influence on AT (B = 0.66, p < .001) and BI (B = 0.17, p = 
.010) so hypotheses H8 and H10 are supported. The influence of AT on BI is also significant (6 
= 0.89, p < .001) so hypothesis H11 is also supported. 

The model explains 79.2% variance in the perceived ease of use (PEU), 39.2% variance 
in the perceived usefulness (PU), 56.0% in the attitude (AT), and 93.1% variance in the intention 
to use. 


Discussion 


This exploratory research contributes with an empirically validated model that explains 
the contribution of the perceived usefulness and attitude to the acceptance of an online teaching 
and learning platform by university students during the pandemic. The model explains a lot 
of variance in the continuance intention towards online learning and brings insights into the 
role played by the three external variables: Ease of access, facilitating conditions, and content 
adaptation. 

The findings are similar with the results reported by Pal and Patra (2020) and Mo et al. 
(2021) as regards the contribution of the main predictors of behavioral intention. The content 
adaptation is quasi-similar to the characteristics of the technology in the model of Pal and Patra 
(2020). The difference as regards the variance explained by the model in the behavioral intention 
(much higher in this study) is due to the different contributions of each external variable. 

The ease of access had a significant influence on the perceived ease of use and an indirect 
influence on the attitude and continuance intention. The ease of access, at any time and from 
anywhere, to educational activities and resources is critical for the success of online learning 
and has particular importance under the conditions of exclusive online education during the 
pandemic. Although the facilitating conditions did not have a significant direct influence on the 
perceived usefulness, they proved to be a significant antecedent of the perceived ease of use. 

Content adaptation had a significant influence on both the perceived ease of use and 
perceived usefulness. Adapting the content to the specifics of delivery in the online environment 
is an essential condition for the success of the didactic process, unanimously recognized by 
teachers. According to Debarger et al. (2017), curriculum adaptation involves an intentional 
effort to bring existing materials into line with new visions by adding, adapting, or transforming 
materials. Technology-mediated online teaching-learning involves more than transferring 
content from teacher to students or replicating it as in a face-to-face model, according to 
recommendations proposed by the Columbia Center for Teaching and Learning (https://ctl. 
columbia.edu/resources-and-technology/teaching-with-technology/teaching-online/adapting- 
your-course/). Since online learning requires a high degree of independence and the assumption, 
commitment on the part of the student, the contents must be clear, understandable, attractive, 
creative, useful, relevant, organized in a logical sequence. Games, ice breakers, digital content 
(audio, video, images, articles, web links), structured discussions, reflection assignments 
(https://blog.alo7.com/how-to-adapt-content-for-the-online-classroom/), as well as various 
educational applications and resources can be used as open sources. Content that makes sense 
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to the student, that stimulates their interest, becomes content relevant to their cognitive needs 
and it is perceived as usefulness. 

The influence of attitude is very high which is explained by the specific learning 
conditions during the pandemic. There are several perceived benefits of online learning, 
such as the commodity of ubiquitous access and learning from home, flexibility in learning, 
time and money savings (especially for nonresidential students), and safety (Akuratiya & 
Meddage, 2020; Baczek et al., 2021; Manea et al., 2021). These benefits make students feel 
comfortable using the online learning platform. Distance education also has disadvantages, 
such as concentration difficulties, frustration, lack of motivation, and even stress (Gorghiu et 
al., 2021; Lamanauskas & Makarskaite-Petkeviciene, 2021) which could be compensated if 
online learning is interesting and pleasant, catch students’ attention and keep them engaged 
with the online courses and seminars. This is consistent with the relatively high perceptions of 
attitude items (3.92 and 3.76). 

The overall perception of the online platform is positive which is explained by the 
familiarity of students with the LMS which has been currently used before the pandemic for 
downloading learning content and assignments and uploading homework. It is also explained 
by the fact that the participants are enrolled in a psycho-pedagogical module which may favor 
a better understanding of the benefits of online learning. 

This study has multiple implications for academic staff, as online teaching-learning will 
continue post-pandemic, alternating or complementing the traditional format. The analyzed 
variables show that learning becomes relevant for the student in the online environment when 
conditions are met such as content adaptation, use of appropriate LMS, and perceived online 
learning usefulness has a positive impact on learners’ attitude toward e-learning (also confirmed 
by Um, 2021). 

Also, considering the online learning situations, the students have more responsibility 
for achieving academic success than in the context of conventional face-to-face classes. Thus, 
a proactive, self-managing attitude and behavior in the learning process can stimulate intrinsic 
motivation for one’s training. 

The three rejected hypotheses suggest reconsidering the TAM-based conceptualization 
in future studies. It seems that ease of use is less and less relevant for the intention to use an 
e-learning platform which explains why several studies did not test a direct path from PEU to 
BI. On another hand, exploring antecedents of perceived usefulness is a promising research 
direction. 

As this represents an exploratory study, there are several inherent limitations. The first 
limitation is related to the small number of items of five constructs. In the near future, the scale 
will be revised accordingly. The second limitation is related to the sample size, which is rela- 
tively small. Also, the participants are from only one university. Another limitation is related 
to the external variables: although the existing research shows a plethora of antecedents of the 
main drivers of the intention to use, this study included only three. Future work will investigate 
other external variables that are relevant to the context of use and the purpose of the research. 


Conclusions 


Understanding the main factors that contribute to the integration of e-learning systems 
represents a pre-condition for making the best decisions in education. In this respect, the 
collected results open new perspectives on the factors that can contribute to the achievement 
of an effective didactic process in the online environment. It is imperative that teachers know 
and create relevant contexts for students to perform both in the classroom environment, but 
also online. In an education system that values the paradigm of student-centered education, the 
concern for optimizing the educational process must be continuous. 
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The integration of modern technological means in education, the use of platforms 
to deliver learning content, and unlimited access in time and space to valuable educational 
resources, are not the exclusive result of the pandemic. In an academic environment, all of 
those have existed and worked before. The merit of the pandemic is to open unimaginable 
opportunities, accelerate transformative processes in education, and open minds and visions. 

The high influence of the attitude on the continuance intention shows that university 
students got used to learning from home and suggests extending the model in future research by 
including specific antecedents of the main determinants. 
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